Systems and methods for computing applications

ABSTRACT

Systems and methods for dynamic development and/or deployment of computing applications including a development framework, a visual design subsystem, and a deployment subsystem, where at runtime the deployment subsystem is operable to dynamically deploy a computing application realized by a blueprint by sending a request at runtime for graphs and components instantiated by the blueprint.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/837,670 filed Aug. 27, 2015 which is a continuation of U.S. patentapplication Ser. No. 14/343,299 filed Apr. 24, 2014 which is a NationalPhase Entry of PCT application PCT/CA2012/000820 filed Sep. 6, 2012which claims the benefit of U.S. Provisional Patent Application No.61/531,953 filed Sep. 7, 2011 and U.S. Provisional Patent ApplicationNo. 61/598,670 filed Feb. 14, 2012 all of which are hereby incorporatedby reference in their entireties.

FIELD

The described embodiments relate to systems and methods for computingapplications, and in particular, to systems and methods for dynamicdevelopment and deployment of computing applications.

BACKGROUND

Computing applications generally involve processing data, performingoperations on the data to carry out specific functions, completingtasks, controlling components, and so on. An example computingapplication is a media application. Media applications generally involveproducing, transforming or delivering media data, or a combinationthereof. New devices and technology increase the use of computingapplications and data. New network capabilities and improved data accessfurther increase the use of computing applications and data. Theavailability of multiple computing languages, protocols and platformsincrease options available to computing application providers,developers, and users but may make it difficult to use a combination ofmultiple computing applications or combine a new computing applicationwith an existing system or architecture due to integration,interoperability and connectivity problems. There exists a need forimproved methods and systems for the development and deployment ofcomputing applications, or at least alternatives.

SUMMARY

In a first aspect, embodiments described herein provide a system fordynamic development of computing applications comprising a developmentframework, one or more processors, and a memory coupled to the one ormore processor and configured to store instructions executable by theone or more processors to configure the development framework to definecomponents and graphs, wherein each component defines a computingprocessing mechanism for processing data containers of computing data atapplication runtime, wherein each graph identifies components,connections between the components, and properties for the components,wherein a graph is an instantiation of a corresponding blueprint atapplication runtime, wherein the development framework enablescomponents to be embedded within other components. Additional andalternative functionality is described herein.

In accordance with some embodiments, a graph may deliver functionalitydefined by the components identified by the graph, and wherein ablueprint connects the functionality to a running environment. Theblueprint may provide business logic for the corresponding graph.

In accordance with some embodiments, the system may further comprise avisual design subsystem for realizing computing applications, whereinthe visual design subsystem is operable to arrange components intofunctional blocks, define specific orders of operation for thefunctional blocks, and define connections between the functional blocksto instantiate the computing applications. Additional and alternativefunctionality is described herein.

In accordance with some embodiments, each component may be associatedwith one or more versions, wherein at least one of a graph and ablueprint comprises a reference to a solution set of components, whereinthe solution set identifies a version for each component.

In accordance with some embodiments, at least one component may beassociated with one or more versions and wherein the developmentframework enables loading of an appropriate version of the at least onecomponent at application runtime.

In accordance with some embodiments, a first component may be in a firstlanguage and a second component may be in a second different language,wherein the first and second components comprise data and are operableto access the memory and data structures, and wherein the system furthercomprises a translation module operable to translate multiple languagesinto a common language by translating the first and second componentdata and how the first and second component re operable to access thememory and the data structures.

In another aspect, embodiments described herein may provide a method fordynamic development of computing applications: providing a dynamicdevelopment of computing applications comprising a developmentframework, one or more processors, and a memory coupled to the one ormore processor and configured to store instructions executable by theone or more processors to configure the development framework to definecomponents and graphs, wherein each component defines a computingprocessing mechanism for processing data containers of computing data atapplication runtime, wherein each graph identifies components,connections between the components, and properties for the components,wherein a graph is instantiated by a corresponding blueprint atapplication runtime; wherein the development framework enablescomponents to be embedded within other components; developing componentsand graphs for a blueprint; and storing the components and the graphsfor the blueprint in the repository for loading at application runtime.

In another aspect, embodiments described herein may provide a system fordynamic deployment of computing applications comprising: a deploymentsubsystem for deploying computing applications at runtime, one or moreprocessors, and a memory coupled to the one or more processor andconfigured to store instructions executable by the one or moreprocessors to configure the deployment subsystem with a repository,cloud agent, cloud engine, wherein the computing applications arerealized by blueprints, wherein each blueprint may be used toinstantiate a graph at application runtime, wherein a graph identifiescomponents, connections between the components, and properties for thecomponents, wherein each component defines a computing processingmechanism for processing data containers of computing data atapplication runtime, wherein each graph identifies components, whereinthe repository stores the graphs and the components for loading atapplication runtime, wherein the cloud agent controls at least one cloudengine, wherein the cloud engine provides a running environment for thecomputing application by using blueprints to instantiate graphs atapplication runtime; wherein at runtime the deployment subsystemdynamically constructs and deploys a computing application by sending arequest at runtime to the repository for the graphs instantiate bycorresponding blueprints and components identified therein. Additionaland alternative functionality is described herein.

In accordance with some embodiments, each component may be associatedwith one or more versions, wherein at least one of a blueprint and agraph comprises a reference to a solution set of components, wherein thesolution set identifies a version for each component.

In accordance with some embodiments, the system may further comprise alicense server, wherein the license server may dynamically managelicenses and associates licenses with components and graphs, wherein useof components and graphs at application runtime requires the appropriatelicense.

In accordance with some embodiments, the system may further comprise ajob manager, wherein the job manager dispatches blueprints and graphs tocloud agents based on available licenses managed by the license server.The job manager may also be configured to provide job and cloud enginedispatch, failover, tracking and reporting.

In accordance with some embodiments, the system may further comprise asecurity manager, wherein the security manager provides for secureconnections and communications between system components. Additional andalternative functionality is described herein.

In accordance with some embodiments, each graph identifies components,connections between the components, and properties for the components,wherein components are connected by different types of pins.

In accordance with some embodiments, a data container defines a datatype and a data object, wherein the data type is metadata describing thedata container and the data object maintains raw data.

In accordance with some embodiments, the repository manages versioningof components and graphs to keep track of updates made thereto, whereinthe repository serves the components and graphs at application runtimeusing appropriate versions of the graphs and components. Additional andalternative functionality is described herein.

In accordance with some embodiments, the cloud agent is provided to eachuser system to manage the local resources of the user system, whereinthe cloud agents interact with cloud engines to instantiate graphs usingblueprints. Additional and alternative functionality is describedherein.

In accordance with some embodiments, the system may further comprise anormalization module operable to receive input data files and convertand parse the input data files into data containers for processing by agraph.

In accordance with some embodiments, the system may further comprisecode signing module operable to digitally sign each component toassociate a developer, license, or both with at least one component.Additional and alternative functionality is described herein.

In accordance with some embodiments, the system may further comprise adigital certificate associated with a component provider subsystem,wherein the component provider subsystem provides one or morecomponents; a digital certificate associated with a user computingsubsystem, wherein the user computing subsystem is associated with acomputing application, wherein the computing application involves acomponent provided by the component provider computing system; a licenseserver configured to digitally sign a component by linking the componentto the digital certificate associated with the user computing subsystemand the digital certificate associated with the component providersubsystem to indicate that the user computing system and the componentprovider subsystem accept performance of the digitally signed component;wherein at runtime prior to deploying each component the deploymentsubsystem queries the license server to determine whether the componentis linked to the digital certificate associated with the user computingsubsystem and the digital certificate associated with the componentprovider subsystem.

In accordance with some embodiments, the deployment subsystem may befurther configured to partition a graph into two or more subgraphs andhandle interprocess communications between the two or more subgraphs.

In another aspect, embodiments described herein may provide a method fordynamic deployment of computing applications: providing a deploymentsubsystem for deploying computing applications at runtime, one or moreprocessors, and a memory coupled to the one or more processor andconfigured to store instructions executable by the one or moreprocessors to configure the deployment subsystem to comprise arepository, cloud agent, cloud engine, wherein the computingapplications identify blueprints, wherein each blueprint may be used toinstantiate a graph at application runtime, wherein a graph identifiescomponents, connections between the components, and properties for thecomponents, wherein each component defines a computing processingmechanism for processing data containers of computing data atapplication runtime, wherein each graph identifies components; storingcomponents and graphs in the repository for loading at applicationruntime; providing, by the cloud engine, a running environment for thecomputing application by using blueprints to instantiate graphs atapplication runtime; controlling, by the cloud agent, the cloud engine;at application runtime, dynamically deploying a computing application bysending a request at runtime to the repository for the graphs andcomponents identified in the blueprint.

In accordance with some embodiments, the method may further compriseproviding a digital certificate associated with a component providersubsystem, wherein the component provider subsystem provides one or morecomponents; providing a digital certificate associated with a usercomputing subsystem, wherein the user computing subsystem is associatedwith a computing application, wherein the computing application involvesa component provided by the component provider computing system;providing a license server configured to digitally sign a component bylinking the component to the digital certificate associated with theuser computing subsystem and the digital certificate associated with thecomponent provider subsystem to indicate that the user computing systemand the component provider subsystem accept performance of the digitallysigned component; receiving, at a license server, acceptance of thecomponent provided by the component provider subsystem in the computingapplication associated with user computing system by receiving thedigital certificate from the user computing subsystem and the digitalcertificate from the component provider computing system; linking, atthe license server, the component provided by the component providersubsystem in the computing application associated with user computingsystem to the digital certificate from the user computing subsystem andthe digital certificate from the component provider computing system;and at application runtime prior to deploying each component, queryingthe license server to determine whether the component is linked to thedigital certificate associated with the user computing subsystem and thedigital certificate associated with the component provider subsystem.

In another aspect, some embodiments described herein provide a systemfor dynamic development and deployment of computing applications (suchas e.g. a media application) comprising: a development frameworkcomprising a software development kit, components, data containers,pins, and graphs, wherein the software development kit is used to definecomponents, graphs, data containers, and blueprints. Each component maydefine a computer processing mechanism for processing data containers atapplication runtime. Each graph may be a template of a set ofcomponents, and each blueprint may be an embodiment of a graph; a visualdesign subsystem configured to output graphs and blueprints to developcomputing applications using components, compound components and othergraphs, wherein the visual design subsystem is operable to arrangecomponents into functional blocks and define specific orders ofoperation for the functional blocks; a deployment subsystem fordeploying computing applications at runtime comprising a repository,cloud agent, and cloud engine. The computing applications identifygraphs, blueprints compound components, and components. The repositoryis configured to store graphs and components for loading at applicationruntime. The cloud engine provides a running environment for graphs andexecutes graphs at application runtime to instantiate computingapplications. The cloud agent may control and manage the cloud engine.At runtime the deployment subsystem may dynamically construct and deploya computing application by sending a request at runtime to therepository for the graphs, compound components, and componentsidentified in the computing application. The deployment subsystem isoperable to deploy a computing application by, at runtime, retrieving,transferring, downloading, and so on, the graphs, blueprints, etc. fromthe repository. The components, graphs, blueprints may not be present inthe computing application and they may be pulled dynamically to createthe computing application at runtime. They may also pre-exist locallydue to previous availability (cache) or through user intent (e.g. a jobmanager persists the availability because the job is going to repeat).

In accordance with some embodiments, the deployment subsystem mayfurther comprise a license server which may dynamically manage licensesand associate licenses with components and graphs. Use of components andgraphs identified in a computing application requires the appropriatelicense.

In accordance with some embodiments, the deployment subsystem mayfurther comprise a job manager, which dispatches cloud engines based onavailable licenses managed by the license server.

In accordance with some embodiments, the deployment subsystem mayfurther comprise a security manager which provides for secureconnections and communications between system components.

In accordance with some embodiments, the deployment subsystem mayfurther comprise a job manager configured to provide job and enginedispatch, failover, tracking and reporting. The job manager may also beconfigured to provide the highest level of access to the running cloudengines, and provide centralized access to the cloud engines regardlessof state (running or not). The job manager may further self-extendinterfaces (e.g. web services) based on the graph/blueprint that isloaded on the cloud engine to provide a namespace (similar to the web)which may allow the developer to discover which graphs/components areused in that particular application, query/set parameters, and so on.

In accordance with some embodiments, a graph may define a set ofcomponents, where components in the set are connected by different typesof pins.

In accordance with some embodiments, a data container may define a datatype and a data object, wherein the data type is metadata describing thecontainer and the data object maintains raw data.

In accordance with some embodiments, the repository is operable tomanage versioning of components and graphs to keep track of updates madethereto. The repository serves the components and graphs at applicationruntime using appropriate versions of the graphs and components.

In accordance with some embodiments, the cloud agent is provided to eachuser system to manage the local resources of the user system. The cloudagents interact with cloud engines to execute graphs in order to runcomputing applications.

In accordance with some embodiments, the system may further comprise anormalization module operable to receive input files and convert andparse the input files into data containers to be processed by a graph.

In accordance with some embodiments, the system may further comprise acode signing module operable to digitally sign each component toassociate a developer/license with each component.

In accordance with some embodiments, the system may further comprise atranslation module operable to translate multiple languages into acommon language for system.

In accordance with another aspect, embodiments may provide a method fordynamic development and deployment of computing applications (such asmedia applications, for example) comprising: providing a developmentframework comprising a software development kit, components, datacontainers, pins, and graphs. The software development kit may be usedto define the components, graphs, and data containers. Each componentmay define a computer processing mechanism for processing datacontainers of media data at application runtime. Each graph may define aset of components, along with specific connections between thecomponents and properties for the components; providing a visual designsubsystem to define and output graphs, wherein graphs may be used torealize and create computing applications; using the visual designsubsystem to arrange components into functional blocks and definespecific orders of operation for the functional blocks, includingconnections between the functional blocks; providing a deploymentsubsystem for deploying computing applications at runtime comprising arepository, cloud agent, and a cloud engine. The computing applicationsmay identify graphs, compound components, and components. The cloudengine may provide a running environment for graphs and executes graphsat application runtime to instantiate computing applications. The cloudagent controls the cloud engine; storing components and graphs output bythe visual design subsystem in the repository for loading at applicationruntime; and at application runtime, dynamically constructing anddeploying a computing application by sending a request at runtime to therepository for the graphs, compound components, and componentsidentified in the computing applications.

In another aspect, embodiments described herein provide a system fordynamic development and deployment of computing applications comprising:a development framework comprising a software development kit,components, data containers, pins, and graphs. The software developmentkit may be used for defining the components, graphs, blue prints, anddata containers. Each component may define a media processing mechanismfor processing data containers of computing data at application runtime.Each component may be associated with one or more versions. Each graphmay be a template identifying components and used to generate acorresponding blueprint. A blueprint may be a final embodiment of thegraph and may comprise a reference to a solution set of components,where the solution set of components may identify a version for eachcomponent; a visual design subsystem configured to define and outputgraphs in order to realize and create computing applications. The visualdesign subsystem may be operable to arrange components into functionalblocks and define specific orders of operation for the functionalblocks, including connections between the functional blocks. The visualdesign subsystem may be further configured to define a solution set ofcomponents by identifying a version of each component; a deploymentsubsystem for deploying computing applications at runtime comprising arepository, one or more cloud agents, and one or more cloud engines. Thecomputing applications identify graphs, compound components, andcomponents. The repository may be configured to store graphs, blueprintsand components for loading at application runtime. The cloud engine mayprovide a running environment for graphs and may execute blueprints ofthe graphs at application runtime to instantiate computing applications.The cloud agent may be operable to control the cloud engine; wherein atruntime the deployment subsystem dynamically constructs and deploys acomputing application by sending a request at runtime to the repositoryfor the graphs, blueprints, compound components, and components,including appropriate versions thereof, identified in the computingapplications.

In a further aspect, embodiments described herein provide method fordynamic development and deployment of computing applications: providinga development framework comprising a software development kit,components, data containers, pins, and graphs. The software developmentkit may define components, graphs, blueprints, and data containers. Eachcomponent may define a computer processing mechanism for processing datacontainers of computing data at application runtime. Each component isassociated with one or more versions. Each graph may be a templateidentifying components and may be used to generate a correspondingblueprint. A blueprint may be a final embodiment of the graph and maycomprise a reference to a solution set of components, where the solutionset of components identifies a version for each component; providing avisual design subsystem for defining and outputting graphs in order todevelop media applications, and for defining a solution set ofcomponents by identifying a version of each component; using the visualdesign subsystem to define a graph by arranging components intofunctional blocks and defining specific orders of operation for thefunctional blocks, including connections between the functional blocks,where the graph references a solution set of components; using thevisual design subsystem to define a solution set of componentsreferenced by the graph by receiving a selected version for eachcomponent in the solution set of components; providing a deploymentsubsystem for deploying computing applications at runtime comprising arepository, one or more cloud agents, and one or more cloud engines. Themedia applications identify graphs, compound components, and components.The cloud engine provides a running environment for graph and executesblueprints of the graphs at application runtime to instantiate computingapplications The cloud agent may be operable to control the cloudengine; storing components and graphs output by the visual designsubsystem in the repository for loading at application runtime; and atapplication runtime, dynamically constructing and deploying a computingapplication by sending a request at runtime to the repository for thegraphs, compound components, and components, including appropriateversions thereof identified in the computing application.

In another aspect, embodiments described herein provide a system fordynamic development and deployment of computing applications comprising:a development framework comprising a software development kit,components, data containers, pins, and graphs. The software developmentkit may be for defining the components, graphs, blue prints, datacontainers. Each component may define a media processing mechanism forprocessing data containers of computing data at application runtime.Each graph may be a template identifying components and may be used togenerate a corresponding blueprint. A blueprint may be a finalembodiment of a graph, and a graph is the instantiation of acorresponding blueprint at application runtime; a digital certificateassociated with a component provider subsystem, where the componentprovider subsystem provides one or more components of the developmentframework;

a digital certificate associated with user computing subsystem, wherethe user computing subsystem is associated with a computing application,where the computing application involves a component provided by thecomponent provider computing system; a visual design subsystemconfigured to define and output graphs in order to develop computingapplications, where the visual design subsystem is operable to arrangecomponents into functional blocks and define specific orders ofoperation for the functional blocks; a deployment subsystem fordeploying computing applications at runtime comprising a repository, oneor more cloud agents, and one or more cloud engines. The computingapplications identify graphs, compound components, and components. Therepository may be configured to store graphs and components for loadingat application runtime. The cloud engine may provide a runningenvironment for graph and may execute blueprints of the graphs atapplication runtime to instantiate computing applications. The cloudagent is operable to control one or more cloud engines. The deploymentsubsystem may further comprise a license server configured to digitallysign a component by linking the component to the digital certificateassociated with the user computing subsystem and the digital certificateassociated with the component provider subsystem to indicate that theuser computing system and the component provider subsystem acceptperformance or conformity of the digitally signed component, whereperformance may relate to runtime performance, service level agreement,and other measures of performance; wherein at runtime the deploymentsubsystem dynamically constructs and deploys a computing application bysending a request at runtime to the repository for the graphs,blueprints, compound components, and components identified in thecomputing applications. Prior to deploying each component, thedeployment subsystem may query the license server to determine whether acomponent is linked to a digital certificate associated with the usercomputing subsystem and the digital certificate associated with thecomponent provider subsystem.

In a further aspect, embodiments described herein provide a method fordynamic development and deployment of computing applications comprising:providing a development framework comprising a software development kit,components, data containers, pins, and graphs. The software developmentkit may define components, graphs, blueprints, and data containers. Eachcomponent may define a computing processing mechanism for processingdata containers of computing data at application runtime. Each componentmay be associated with one or more versions and each graph may be atemplate identifying components and may be used to generate acorresponding blueprint. A blueprint may be a final embodiment of agraph, and a blueprint may be used to instantiate a graph at applicationruntime; providing a digital certificate to a component providercomputing system, where the component provider computing system providesone or more components to the development framework; providing a digitalcertificate to a user computing subsystem, where the user computingsubsystem is associated with a media application, and where thecomputing application involves a component provided by the componentprovider computing system; providing a visual design subsystem fordefining and outputting graphs in order to develop computingapplications, and for defining a solution set of components byidentifying a version of each component; using the visual designsubsystem to define a graph by arranging components into functionalblocks and defining specific orders of operation for the functionalblocks, where the graph may reference a solution set of components;providing a deployment subsystem for deploying computing applications atruntime comprising a repository, one or more cloud agents, and one ormore cloud engines. The computing applications may identify graphs,compound components, and components. The cloud engine may provide arunning environment for graphs and may execute blueprints of the graphsat application runtime to instantiate computing applications. The cloudagent is operable to control the cloud engine; receiving, at a licenseserver, acceptance of the component provided by the component providersubsystem in the computing application associated with the usercomputing system by receiving the digital certificate from the usercomputing subsystem and the digital certificate from the componentprovider computing system; linking, at the license server, the componentprovided by the component provider subsystem in the computingapplication associated with user computing system to the digitalcertificate from the user computing subsystem and the digitalcertificate from the component provider computing system; storingcomponents and graphs output by the visual design subsystem in therepository for loading at application runtime; and at applicationruntime, dynamically constructing and deploying the computingapplication associated with the user computing subsystem by sending arequest at runtime to the repository for the graphs, compoundcomponents, and components identified in the computing application; andprior to deploying the component provided by the component providercomputing system, querying the license server to determine whether thecomponent is linked to the digital certificate associated with the usercomputing subsystem and the digital certificate associated with thecomponent provider subsystem.

Variations and combinations may also be provided by the embodimentsdescribed herein. Additional aspects of various example embodiments areidentified and described in the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of embodiments of the systems and methodsdescribed herein, and to show more clearly how they may be carried intoeffect, reference will be made, by way of example, to the accompanyingdrawings in which:

FIG. 1A illustrates a block diagram of the system for dynamicdevelopment and deployment of computing applications, in accordance withan example embodiment;

FIG. 1B illustrates a block diagram of the data flow of a system fordynamic development and deployment of computing applications, inaccordance with an example embodiment;

FIG. 1C illustrates another block diagram of the data flow of a systemfor dynamic development and deployment of computing applications, inaccordance with an example embodiment;

FIG. 2 illustrates a block diagram of example components in accordancewith an example embodiment;

FIG. 3 illustrates a block diagram of example properties of an examplecomponent in accordance with an example embodiment;

FIG. 4 illustrates a block diagram of example data container andcomponents in accordance with an example embodiment;

FIG. 5 illustrates a block diagram of an example graph in accordancewith an example embodiment;

FIG. 6 illustrates a block diagram of an example interface for a visualdesign subsystem in accordance with an example embodiment;

FIG. 7 illustrates a block diagram of an example interface for arepository in accordance with an example embodiment;

FIG. 8 illustrates a block diagram of an example interface for a jobmanager in accordance with an example embodiment;

FIGS. 9 and 10 illustrate block diagrams of example web servicesimplementations in accordance with example embodiments;

FIGS. 11 and 12 illustrate block diagrams of example implementations ofan asset management and publishing system in accordance with exampleembodiments;

FIG. 13 illustrates a block diagram of an example interface for defininga solution set of components in accordance with example embodiments;

FIG. 14 illustrates a block diagram of an example certification systemin accordance with example embodiments;

FIG. 15 illustrates a block diagram of dynamic provisioning inaccordance with example embodiments;

FIG. 16 illustrates a block diagram of partitioning mixed architecturesin accordance with example embodiments;

FIG. 17 illustrates an example browser based console to access thelicense server in accordance with example embodiments;

FIG. 18 illustrates a block diagram of stand-alone deployment inaccordance with example embodiments; and

FIG. 19 illustrates a block diagram of network deployment in accordancewith example embodiments.

The drawings, described below, are provided for purposes ofillustration, and not of limitation, of the aspects and features ofvarious examples of embodiments described herein. The drawings are notintended to limit the scope of the teachings in any way. For simplicityand clarity of illustration, elements shown in the figures have notnecessarily been drawn to scale. The dimensions of some of the elementsmay be exaggerated relative to other elements for clarity. Further,where considered appropriate, reference numerals may be repeated amongthe figures to indicate corresponding or analogous elements.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

It will be appreciated that numerous specific details are set forth inorder to provide a thorough understanding of the exemplary embodimentsdescribed herein. However, it will be understood by those of ordinaryskill in the art that the embodiments described herein may be practicedwithout these specific details. In other instances, well-known methods,procedures and components have not been described in detail so as not toobscure the embodiments described herein. Furthermore, this descriptionis not to be considered as limiting the scope of the embodimentsdescribed herein in any way, but rather as merely describingimplementation of the various example embodiments described herein.

The embodiments of the systems and methods described herein may beimplemented in hardware or software, or a combination of both. However,these embodiments may be implemented in computer programs executing onprogrammable computers, each computer including at least one processor,a data storage system (including volatile and non-volatile memory and/orstorage elements), and at least one communication interface. Forexample, the programmable computers may be a server, network appliance,set-top box, embedded device, computer expansion module, personalcomputer, laptop, personal data assistant, cloud computing system ormobile device. A cloud computing system is operable to deliver computingservice through shared resources, software and data over a network.Program code is applied to input data to perform the functions describedherein and to generate output information. The output information isapplied to one or more output devices to generate a discernible effect.In some embodiments, the communication interface may be a networkcommunication interface. In embodiments in which elements are combined,the communication interface may be a software communication interface,such as those for inter-process communication. In still otherembodiments, there may be a combination of communication interfaces.

Each program may be implemented in a high level procedural or objectoriented programming or scripting language, or both, to communicate witha computer system. However, alternatively the programs may beimplemented in assembly or machine language, if desired. In any case,the language may be a compiled or interpreted language. Each suchcomputer program may be stored on a storage media or a device (e.g. ROMor magnetic diskette), readable by a general or special purposeprogrammable computer, for configuring and operating the computer whenthe storage media or device is read by the computer to perform theprocedures described herein. Embodiments of the system may also beconsidered to be implemented as a non-transitory computer-readablestorage medium, configured with a computer program, where the storagemedium so configured causes a computer to operate in a specific andpredefined manner to perform the functions described herein.

Furthermore, the system, processes and methods of the describedembodiments are capable of being distributed in a computer programproduct including a physical non-transitory computer readable mediumthat bears computer usable instructions for one or more processors. Themedium may be provided in various forms, including one or morediskettes, compact disks, tapes, chips, magnetic and electronic storagemedia, and the like. The computer useable instructions may also be invarious forms, including compiled and non-compiled code.

Embodiments described herein may relate to various types of computingapplications, such as media applications, resource related applications,voting applications, user registration applications, integritymanagement applications, and so on. By way of illustrative exampleembodiments may be described herein in relation to media applications.

Referring now to FIG. 1A, there is shown a block diagram of a system 10for dynamic development and/or deployment of computing applications inaccordance with an example embodiment. By way of example, a computingapplication may be a media application. A media application may be acomputing application designed to perform specific tasks and activitiesfor manipulating media data using a combination of hardware and softwarecomputing components. For example, the media application may involveprocessing media data, performing operations on the data to carry outspecific functions, completing tasks, controlling components, producing,transforming or delivering media data, or a combination thereof. Themedia application may generate a deliverable or transform a deliverablefor provision to output devices and for generation of a discernableeffect, such as by transforming received input media data into adeliverable, for example. The media application may process, transformand manipulate input data streams to generate a complete media programfor display, broadcasting, distribution, and so on. For example,playback of the input data stream may be discernably different fromplayback of the deliverable generated or transformed by the mediaapplication.

The system 10 may scale from simple media applications run on a localcomputer to complex media applications deployed on a cloud computingsystem. A cloud computing system is operable to deliver computingservices through shared resources, software and information over anetwork. The system 10 may be operable for multiple platforms (e.g.Windows, Linux, OS X) and multiple languages (e.g. C++, Java,Scripting), and may use standards based interfaces (e.g. SOAP, XML).

The system 10 may be implemented as a cloud computing system and may beaccessible to users through an external interfaces layer 38 which mayallow integration with existing processes, applications and systems. Thesystem 10 may include a development framework 12 and a visual designsubsystem 30 to define and output graphs 28 in order to develop mediaapplications. The system 10 may include a deployment subsystem 14 fordynamically deploying media applications at runtime. The system 10 mayprovide a platform for building, developing and deploying professionalworkflow applications for desktop, networked and cloud based systems.

By way overview, the development framework 12 may be used for thedevelopment of component 24 and workflow (e.g. graphs 28, blueprints 28a) technologies. The repository 32 may provide a centralized pool ofcomponent 24 technologies and workflow blueprints 28 a and may act asboth the warehouse and supply chain (for syncing upstream/downstreamrepositories 32). The visual designer may be used to design and testworkflow graphs 28 and blueprints 28 a. A license server 42 may controlauthorization of component technologies. The system 10 may provide oneor more of the following features: multi-platform support through theframework SDK 20; multi-language support allows native development inmultiple languages such as C++, Java, and scripting languages; supportfor flexible workflow models with inline, parallel, and staged executionof individual processes; consistent management interfaces throughstandards based web services regardless of workflow complexity or scope;dynamic scalability allows for simple and complex solutions to easilyscale to large volume processing capabilities with low provision anddeployment costs. Other features may also be provided by system 10 asdescribed herein.

The system 10 may enable decomposition of hardware and software problemsinto their core elements. These core elements may be referred to ascomponents 24. By breaking down multiple problems, a catalog ofcomponents 24 may be developed that can be brought together in differentways (e.g. by graphs 28, blueprints 28 a) to solve new problems. Forexample, a user may want to perform video compression and send emailnotification upon completion. These two problems are very different butby combining elements of the video compression problems, that is,components 24 for video codec, multiplexer and file writer; and theemail problem, that is, components 24 for database lookup, reportgenerator and email engine; system 10 can combine the two into acomplete solution that not only performs the core video compression, butmay also sends out notification emails to the users who need to benotified of the project completion.

The system 10 may enable registration of the components 24 into arepository 32 of technology, used to store, manage, and access thesetechnologies in a controlled environment that may be centralized ordistributed. The system 10 may allow these repositories 32 to be chainedtogether into managed supply chains, where downstream repositories 32can be synced with upstream repositories 32.

The system 10 may control access to components 24 using a floatinglicense server 42 which may check out licenses when components 24 arebeing used.

The system 10 may provide a communication/management bridge between ahigher level application and the cloud engines 36 a which run jobs. Thecloud agents 34 may provide that bridge. The cloud agents 34 may providea consistent web services integration and management point regardless ofthe complexity of the solution.

The system 10 may provide a method for creating workflow solutions(graphs 28, blueprints 28 a) from components 24. The visual designer 30may be implemented as a visual design tool which allows new solutions tobe created, tested, and saved as graphs 28 or blueprints 28 a that canbe referenced by an application. Blueprints 28 a can also be stored inthe repository 32, becoming part of the managed supply chain.

The system 10 may run cloud engines 36 to execute jobs. When theapplication sends a command to the cloud agent 34 to run a job, thecloud agent 34 determines which components are required to start runningthe solution and acquires those components from the repository 32. Forexample, the cloud agent 34 creates the cloud engine 36 a, the cloudengine 36 a loads the graph 28 or blueprint 28 a, acquires the requiredlicenses from the license server 42, and runs the job. The cloud engine36 a may dynamically acquire new component licenses on the fly asrequired by the currently running workflow. When the job is complete thelicenses are returned to the pool of licenses managed by the licenseserver 42.

Development Framework

The development framework 12 may include components 24, compoundcomponents 26 (components embedded within other components), datacontainers 56, graphs 28, and blueprints 28 a. The development framework12 may be accessible through a software development kit (SDK) 20, webservices or by using the visual design subsystem 30. Both the SDK 20 andthe visual design subsystem 30 may be used to develop components 24,compound components 26, and graphs 28, and to define new data types todefine new data containers 56. System 10 may provide a graph-basedprocessing engine, where graphs 28 are made up of reusable components 24with discrete processing functions. The system 10 may also provide theframework for development and deployment of graphs 28 and components 24.As noted above, the development framework 12 may be accessible throughweb services, where a user may create graphs and components using theweb services application programming interface.

As noted, the SDK 20 may be used to define components 24, graphs 28,data containers 56, and other features of the system 10. The SDK 20 maybe language independent. An example of the SDK 20 in java is:

public class LoggingComponent extends JavaComponent {  @Override  publicvoid process(DataContainer data, String inputPinName ) {  super.process(data, inputPinName);   log( Level.INFO, ″Process: ″+data );   getOutputPin( ).process(data);  } }

Other languages may also be used and this is an example only.

The SDK 20 may include a framework API and a component API.

The framework API may be used to create the graphs 28 used by anapplication, to control loading and executing graphs 28 and to providestatus to the application.

The component API may be used to create individual components 24.

Separating the component API from the framework API may allow componentdevelopers to focus on the development of a component 24 withoutrequiring knowledge about the logistics of the whole environment.

The SDK 20 may include application programming interface in multiplelanguages (such as java, C++ for example) to create components.Components may be created using one or more languages.

Components 24 are building blocks of the system 10. A component 24 is anobject, plug in or set of software code that defines a processingmechanism and uses the SDK 20 to interact with the development framework12. At application runtime, a component 24 is configured to process dataflowing through the component 24 as a data container 56. Each component24 may be a single component 24 or a compound component 26 built up ofmultiple embedded components 24. A component 24 may contain plug infiles, and other files, such as jar files, dlls, and so on.

Referring now to FIG. 2 there is shown a block diagram of examplecomponents 24 in accordance with an example embodiment. Examples ofcomponents 24 for a media application context include video input 24 a,video process 24 b, file sink 24 c, logic branch 25 (decisioning androuting based on criteria, such as for example video format), stripletterbox 24 d, and aspect ratio 24 e. Other examples of components 24are shown in FIGS. 5 and 6, such as file source 24 f, interlace detector24 g, film removal 24 h, deinterlacer 24 i, noise reduction 24 j, buffer24 k, file sink 24 x, YUV to RGB 24 m, image converter 24 n, flowcontrol 24 o, file input 24 u, color space converter 24 v, scaler 24 w,and AVC encoder 24 y.

Components 24 may have properties and values for those properties. Acomponent's 24 properties configure its behavior. The properties provideruntime information, transient information, results, and so on.Referring now to FIG. 3 there is shown a table representing exampleproperties 25 and values 27 of an example component 24 (file source 24s) in accordance with an example embodiment. Examples of properties 25include description, configuration warning, default input pin, defaultoutput pin, description, file, name, last error, log, progress, readbuffer size, and so on. Each property may have an associated value 27. Acomponent's 24 properties 25 may be set and modified through the visualdesign subsystem 24 or other interface.

Properties modify the way a component 24 behaves, such as how itprocesses data or the type of output it produces. For instance,properties can be used to change the format of a component's 24 outputor provide scaling information. Properties can be thought of as instancevariables for components 24. Properties can be exposed to othercomponents 24 through pins.

Property attributes may change the way a property is used, exposed andsaved. They may be set in the property declaration in the plugin.xmlfile. For example, properties may have one or more of the followingattributes: transient (when a graph is saved to file, property valuesmay be saved with it by default, however, if a property is transient,its value may not be saved and the default value may be used when thegraph is loaded from file), required (the property must be set for thecomponent to run), hidden (the property is used internally by acomponent and may not be visible to a user), advanced (the propertygenerally does not need to be modified by the user, but may be ofinterest to experienced users), interprocess (the property may beaccessible to processes that are spawned by its graph), and so on.

Properties can be exposed on pins. A property exposed on a pin can bewritten to or read by another component. The name of a property pin isthe same as the name of the property defined in the propertydeclaration. The property pin's display name in the visual designer 30may be the same as the property name unless a pin display name isprovided. Properties may be declared within component definitions in theplugin file, such as a plugin .xml file for example.

Example attributes of properties may include property name, displayname, description, required, advanced, hidden, transient, interprocess,value type, and initial value. A property may be defined as advanced ifit generally does not need to be modified, but may be of interest toexperienced users. For example, setting the attribute advanced=“true”may hide the property in the visual designer 30. The property may becomevisible when an “advanced” box is selected in the visual designer 30.Setting hidden=“true” may hide the property in the visual designer 30.The property may become visible when a “hidden” box is selected in thevisual designer 30. When a graph is saved to file, the property valuesof its components may also be saved. Setting transient=“true” may resultin the property value not being saved. The default property value may beused when the graph is loaded from file. Setting interprocess=“true” maymake a property accessible to processes spawned by its graph. A propertyinitial value may be the default value of the property when thecomponent is initially instantiated.

Property values may be restricted. For example, property values may berestricted to strings, numbers, integers, range of numbers defined by aminimum and a maximum, and so on. Property value restriction attributesmay include a minimum value, a maximum value, a list of enumeratedvalues, data type for values, and so on.

An attribute property initial value may set the initial value of aproperty that has been declared elsewhere (for example, if a propertyhas been declared in an inherited component, in the component's sourcecode or in the framework itself). An example use of property initialvalue may be for instructing a C++ or dual Java/C++ component how toconstruct its native peer.

Components 24 may be language independent and may be developed bydifferent developers in different languages, and used together to createa graph 28, blueprint 29 or compound component 26.

Each component 24 may have one or more versions. A version is a specificform or state of the component 24, and may reflect new developments orimplementations of the component 24. Each version of a component 24 maybe referenced using a version name or number. For example, each versionmay be assigned a number in increasing order. As will be explainedherein in relation to FIG. 13, the system 10 maintains versioning tokeep track of and use different versions of components 24 in graphs 28,blueprints 28 a, and compound components 26. This may result in moreflexible system 10 as different versions of the same component 24 may beusable by graphs, media applications and users, and each are notrequired to use the same component and version thereof.

Components 24 may be written for different architectures or contexts,such as 32 bit and 64 bit architectures. As will be explained herein,system 10 is operable to develop and deploy an application instancewhich combines components written for both 32 bit and 64 bitarchitectures. For example, system 10 is operable to detect whether aparticular media application has been developed using both components 24for 32 bit architectures and components 24 for 64 bit architectures. Ifso, system 10 is operable to create a separate process space or instancefor each context and handle inter process communications using mappingand a shared memory. For example, the system 10 is operable to create a32 bit architecture process instance and a 64 bit architecture processinstance and manage communications between the process instances.

Further, components 24 may be self-contained and isolated from adependency point of view. The entire dependency set of a component 24may be self-contained, being specified and packaged in the componentdistribution unit (e.g. plugin). The component 24 dependencies may alsobe isolated, referring exclusively to the specific component 24 andversion(s) they depend on. This may enable the system 10 to realizecomplex workflows while resolving components 24 dependencies withoutuser intervention. Further, the dependency isolation may allow thesystem 10 to provide distinct behavior while executing blueprints builtwith the same components 24 by isolating the different versions of thesecomponents 24 and their dependencies.

Components 24 may have pins. Pins connect to pins on other components24. Pins may be referenced by name. Pins may connect to multiplecomponents 24, which may be referred to as branching. In accordance withsome embodiments described herein, components 24 do not have to be ofthe same type to connect to the same pin.

There may be different types of pins. There may be input pins, such asan input push and input pull, which can be used to decouple components24 with different fundamental architectures. A push pin pushes itsoutput to the next pin and a pull pin calls for data on its input pin.The pin model controls the flow of data between components. There areoutput pins. There are control pins, including event (out), property(in/out), and command (in) pins.

Pins may be used to pass data between components 24. Pins may exposeproperties and events, and may be used to trigger commands. Static pinsmay be defined within the component 24 definition in the plugin.xml fileand may be created every time a component 24 is instantiated. Dynamicpins may be defined within the component 24 source code and may be addedafter the component 24 has been instantiated.

Input and output pins may be defined as default pins. A default pin maynot need to be referred to by name in the component source code. Theremay be only one default input pin and only one default output pin percomponent.

As noted herein, there may be different types of pins. For example, anOUTPUT_PUSH pin is a type of output pin. Data may be sent through anoutput pin to the next component's 24 input pin. INPUT_PUSH andINPUT_PULL are two different types of input pins. When using a pushinput pin, a component may process data as it arrives on the pin. Whenusing a pull input pin, a component 24 may request data from the pin,blocking until data arrives. Pull input pins may be used in situationswhere there is more than one input pin on a component 24 and the datacoming in through each pin needs to be controlled and coordinated.OUTPUT_IO and INPUT_IO pins are further examples. I/O pins act as bothinput and output pins and are typically used to pass data between graphs28 or as output and input pins in compound components 24. A PROPERTY pinmay expose a component's property so that it can be read or modified byother components 24. There may also be EVENT pins. When an event occurs,the data object generated by the event may be encapsulated in a DataContainer 56 that may be pushed onto the event pin. If the eventgenerates a null object, an empty Data Container 56 may be placed on thepin. The propagation of a Data Container 56 on an event pin signals thatthe event has occurred. COMMAND pins act as triggers to call commands oncomponents. When a data container is placed on a command pin, thecommand is executed. The data on the data container may be used by thecommand, but it does not need to be.

Pins may set data types. For example, a data type feature may provideinformation to the end user about a component's expected input andoutput types. This may be useful at graph 28 design time to ensure thecompatibility of connected components. Once the graph 28 starts, datatypes describing the actual data passing between components 24 may beused. A warning may be generated when incompatible pins are connected.The data type of a static pin may be set in the pin declaration using adata type definition name. The data type definition name may take thename of a data type definition, which is a set of key/value pairs thatdescribe features such as image dimensions, audio format, and encodingformat. For example, a pins data type for an input push pin may be setto integer.

The data type definition may be the default data type of an unconnectedinput pin. When components are connected, the input pin may acquire thedata type of the output pin it is connected to. A component's 24 defaultoutput pin may acquire the data type of the component's default inputpin unless the pins have been decorrelated. All other output pins mayuse their own data type definition names to set their data types.

A data type transform may be used to change the default output pin'sdata type. A data type transform may add or remove data type definitionsfrom the output pin's acquired data type. Consider the following examplewhere the default input pin is defined with a data type of integer. Whenthe component is instantiated, the default input pin and the defaultoutput pin may both have the same data type, namely the “integer” datatype. To change the default output pin's data type to string, a datatype transform may be used to remove the “number” data type definition(of which Integer is a subtype) and add the string data type definition.

Data type restrictions may be used to provide more detail about whattypes of input a pin can accept. While setting a data type on a pin mayact a simple data type restriction, setting a formal data typerestriction on the pin can narrow down the type of data that isacceptable. Explicit data type restrictions may override the restrictionimplied by the pin's data type. Data type restrictions may be nestedusing logic operators (AND, OR, NOT). The syntax for data typerestrictions may follow prefix notation. For example, say you want yourpin to accept all numbers that are not integers: numbers AND (NOTinteger), and in prefix notation this may be: AND number (NOT integer).

There may be a pin definition schema. A component's static pins may bedeclared in its definition. Static input, output, event, command andproperty pins may be declared in the component definition. In the caseof event, command and property pins, the name of the pin may need tomatch the name of the event, command or property, respectively. A pin'sdata type may be defined in the plugin.xml file using a data typedefinition name, data type restrictions and data type transforms canalso be set within the plugin.xml file.

A pin definition may include a name, type, default, data type, anddisplay name. For a pin name, upon compiling the plugin.xml, a constantof the form PIN_<name> may be generated. The pin may be referenced insource code using this constant. For example, the constant PIN_file maybe generated for a pin named “file”. Input, output, event and commandpins may be displayed on the component with this name in the visualdesigner 20. Property pins may have a different display name. Thealternate display name may be set in the property declaration. Pins canbe of type, INPUT_PUSH, INPUT_PULL, OUTPUT_PUSH, COMMAND, PROPERTY,OUTPUT_IO, INPUT_IO or EVENT. A default may be used with input or outputpins. There may be only one default output pin and only one defaultinput pin per component. Setting default to “true” may indicate thatthis pin is the default pin to use when referring to this type of pin.The data type definition may define the expected input or output datatype. The pin data type may act as a data type restriction on an inputpin. The display name may be the pin name displayed in the visualdesigner 30. If the display name is set on a property pin for which thedefined property also has a display name, the pin display name mayappear on the component and the property display name may appear as theproperty name.

A data type restriction may be used to restrict the type of input a pincan accept. When an output pin with a data type that does not meet therestriction conditions is connected to the pin, a warning may begenerated. Data type restrictions may override restrictions based on thepin's defined data type. Data type restrictions may be combined usinglogic operators to create complex restrictions. The syntax of the datatype restriction may follow prefix notation. Example restrictionsinclude string, number, integer and so on. Logic operators AND, OR andNOT may be used to create complex data type restrictions.

A data type transform of a default output pin may be set by the defaultinput pin's data type. If the default output pin will be producingoutput which is different than the input pin's data type, a data typetransform may be used to change its data type. The data type transformmay remove parts of a data type definition or entire definitions. It canalso add data type definitions to a pin data type

The set of components that represent a workflow may be saved as a graph28. Data is encapsulated in data containers 56 that may be passedbetween connected components in a graph 28. Data containers 56 mayconsist of raw data as well as meta-information about the data.

Graphs 28 can be created, saved, and loaded programmatically or with thevisual designer 30. The visual designer 30 is a visual design tool forcreating, testing and saving workflows.

Workflows can be saved as graphs 28 or as blueprints 28 a. Blueprints 28a are graphs 28 with attached meta-data. Graphs 28, blueprints 28 a andcomponents 24 may be registered into a repository 32 that is used tostore, manage, and access components 24 and blueprints 28 a in acontrolled environment that may be centralized or distributed.Repositories 32 may be chained together into managed supply chains wheredownstream repositories 32 can be synced with upstream repositories 32.

Access to components 24 may be controlled through a floating licenseserver 42. A component's 24 license may be checked out when it is beingused.

Applications that use graphs 28 may submit their jobs to an agent 34,which may be a web service interface. The agent 34 may acquire thegraph's 28 components 24 from the repository 32 and launch engines 36 torun the jobs. The engines 36 may load the graphs 28 or blueprints 28 a,and may acquire the licenses needed to run the job. Once the job iscomplete, the licenses may be returned to the license server 42.

A component 24 development steps may include one or more of thefollowing: designing the component 24 and its interface, including adetermination of which features and elements the component 24 may need;saving the design that defines the component 24 in a file for use ingraph 28, such as in a plugin.xml file, where the design of thecomponent 24 may also include the features and elements of the component24; writing and storing the component's 24 source code. Componentdefinitions may be used to instantiate components 24 as a component 24definition may contain all the information required to instantiate acomponent 24, including declarations for properties, commands, eventsand static pins.

Examples of properties may include a name, class name, unique identifier(such as a GUID for example), a description and a category. A componentmay be declared with a name, and an example of which is may be a Javaclass name and a unique GUID. Upon compiling the plugin.xml, a constantof the form NAME_<name> may be generated. The component's name may bereferenced in source code using this constant. For example, the constantNAME_AudioMixer may be generated for a component named “AudioMixer”.

The class name property may reference the component constructor class.For example, when writing a Java or Dual component, the component's Javaclass may be used, or when writing a C++ component, may use uniformNativeComponent class.

Each component may have a unique identifier which may be referred to asa GUID. The unique identifier may be for component licensing. Forexample, upon compiling the plugin.xml, a constant of the formGUID_<guid> may be generated. The component's GUID may be referenced insource code using this constant.

The description property may be a description of your component.

The category property may reference the categories to which thecomponent belongs. Categories may be used for grouping components in adisplay such as in the visual designer 30. Each category may be definedwithin its own element. You may create subcategories by separating thecategory names with forward slashes. For example, if you defined thecategory as “Company X/Audio”, the component would appear in the Audiosubcategory of the Company X category.

A component's 24 definition may declare pins (dynamic and static),properties, events, commands, and capabilities. These elements can beused, modified and managed in the component 24 source code. When acomponent's 24 plugin.xml file is compiled, header and jar files may begenerated. These files declare string constants that correspond to thecomponent elements.

A data container 56 holds the media data that flows between components24. The data container 56 may define a data type and data object. Thedata type may be metadata describing the data container 56 and mayinclude key-value pairs of information (e.g. width, height). The datatypes may be configured to implement inherency and hierarchies. Examplesof data types include image dimension (width, height) and pixel format(color space, bits per sample). The data object may be the raw data,such as in the form of a buffer, string, and so on. Examples of dataobjects include file name (string), audio sample (buffer), video frame(buffer), asset XML, and so on.

A data container 46 may include a timestamp in relation to the mediadata stored therein as the data object. Media data packets typicallyneed to be associated with a timeline as they are received and processedto maintain sequencing and timing. Including a timestamp for the mediadata stored in the data container 56 enables non-linearity of processingand decouples the processing of the media data from the timelinetypically associated with media data. A data container 56 may define anormal form for the input data to be processed by graphs 28 andblueprints 28 a. A data container 56 may associate raw data with a datatype so that both the raw data and data type flow as a unit to provideconcurrency, multiprocessing, which may enable the context to switch atthe data container 56 boundaries, and so on. Data containers 56 mayinclude an individual timestamp with reference to the raw data todecouple the raw media data from its state dependent on a timeline. Datacontainer 56 properties may include read only, write only, andread/write. This may be useful if, for example, a data container 56reaches a branch and needs to be duplicated. One data container 56 maybe marked read only so that the contents cannot be modified while aseparate operation is processing the contents of the duplicate datacontainer 56, for example.

Referring now to FIG. 4 there is shown a block diagram of example datacontainer (metadata, raw buffer) 56 a that flows between two components24, video process 24 p and strip letterbox 24 q.

Data may be passed between components 24 as data container 56 objects. Adata container may include a raw data object and a data type objectdescribing the contents of the raw data. A data type object may be acollection of key/value pairs. Its keys may be defined in one or moredata type definitions defined in plugin.xml files, for example. Datatype definitions may describe features such as image dimensions, audioformat, and encoding format. Data type definitions may be inherited andextended.

Data types may consist of a set of keys that accept values of a certaintype. The values of the keys can be of either simple or complex types. Asimple type may be a primitive type such as INTEGER, BOOLEAN or STRING,while a complex type may be a type where the key's value is itself adata type. Data type definitions may be defined in plugin.xml files, forexample. A data type definition may include keys and their value types,and inherited data type definitions. Data type definitions may bedefined using the data type definition schema. A data type definitionmay have attributes or properties such a name (used to reference thedata type definition), comments (which may include information aboutwhat the data type definition refers to, and how and when it should beuse, which may appear in the Data Type frame of the visual designer helpinterface), inherits (a data type definition can inherit and extendother data type definitions), and so on.

Data type definitions should be decoupled from component definitions. Assuch, separate plugin.xml files may be used for data type definitionsand for your component definitions. If you have defined your own datatype definition, you may need to compile its plugin.xml file beforeusing it. Compiling its plugin.xml may generate a header and class filefor each defined data type. These may automatically be exported to anSDK installation.

Data type definitions declare a set of keys used to describe raw data.Key value types are specified in the key definitions. Acceptable valuesfor the keys can be specified explicitly in the declaration. Examplesdefinitions include channel configurations, language, and so on.

Key definitions may have attributes or properties. Examples include:simple type/complex type which indicates whether the key's type is asimple type (the value is a primitive type) or a complex type (the valueis a data type), key name which may be used to reference the keydefinition, key comments which may include a description of whatproperty of the raw data the key refers to, the key's value type whichmay be simple (primitive type), for example INTEGER, STRING, BOOLEAN, orcomplex (a DataTypeDefinition), where this type should agree with thesimpleType/complexType tag, multivalued which indicates that the key mayaccept a list of 0 or more values, such as for example, theaudio_channel_details key may have as many values as there are audiochannels, and enumeration value which enumerates all possible values forthe key. If a key can only have certain acceptable values, these can belisted as enumerationValues. EnumerationValues may be referenced in thesource code using string constants of the form VAL_<key_name>_<value>.For example, the ISO639_1 value of the language standard key may bereferred to by the constant, VAL_language_standard_ISO639_1.

A plugin package may be a grouping of data type definitions andcomponent definitions defined in a plugin.xml file, for example. Aplugin package can consist of more than one component or data typedefinition. The plugin may also contain libraries, header files and javafiles that may be used to load and support its components or data types.

Data type definitions and components may be distributed in pluginpackages. The framework 12 may be shipped with plugin packages.Additional functionality can be added to the framework 12 by purchasingor developing additional plugin packages. Licenses for the framework 12and its components may be included in a license package. A propertiesfile may be included in the SDK 20 package and may be edited to point tothe license server 42.

Components 24, data type definitions and plugin packages can be createdusing the SDK 20. Each component created may have a unique GUID and eachplugin created may need to be signed. A license package may includeGUIDs and a signing key.

Plugin package attributes may be defined within plugin.xml files.Component 24 definitions and data type definitions may be defined withinthe plugin package definition in the plugin.xml file. The plugin packagedefinition may require a name, a unique pluginID, a plugin version, and,optionally, a provider and description. The name is the name of theplugin, the plugin ID is a name that is unique to each plugin (toguarantee uniqueness, the pluginID may be structured using a reversedomain name, for example), a pluginVersion refers to the plugin version,a provider refers to the organization providing the plugin, anddescription provides a description of the plugin. This should includethe type of components or data type definitions that are distributed inthe plugin package.

All plugin packages may be signed. Signing guarantees authorship andprevents unauthorized modification of the plugin package. Signing mayhappen automatically when the plugin.xml file is compiled. Atcompile-time a private key is requested from the license server. Asignature is then generated using this private key. A public key, theplugin certificate, is also generated. When the plugin package isloaded, the certificate is used to verify that the plugin package hasnot been modified from build time. If the plugin package has beenmodified, or if it has not been signed, it will not load.

Plugin packages may be compiled using a gradle tool. Plugin packages,even those which do not contain source code, may be compiled to generateheader files and files that are used to instantiate their components anddata type definitions. Compiling the plugin package automatically signsyour plugin package and installs it in the SDK 20 installation. Theframework 12 may use gradle to build plugin packages. A SDK 20installation may come with template gradle build files (build.gradle)for different types of projects.

A graph 28 may be a template describing a set of components 24(including compound components 26, other graphs 28), the parametervalues for the components 24, and the connections between the pins ofthe components 24. A graph 28 may define a set of components 24 havingspecific connections and have specific properties (with values). A graph28 may define a set of components having specific connections and havingspecific properties. A graph 28 may be referenced within another graphby a label to dereference from the underlying graph 28. This may beuseful for versioning, as described herein.

A blueprint 28 a may be a final embodiment of a graph 28 and mayreference a solution set of components using a label. A blueprint 28 amay be used to instantiate a graph 28 at application runtime, and mayalso meta-data such as include business logic about the graph 28. Ablueprint 28 a may connect the functionality of a graph to a runningenvironment. A solution set of components 24 may be a set of specificversions of components. A blueprint 28 a may form part of the repository32. Blueprints may be viewed as a business level container of graphs 28.A blueprint 28 a may include one or more graphs as part of a single lifecycle of the application, which may be executed nested or in parallel,or multiple graph 28 stages may be executed in serial form, one afterthe other. A blueprint 28 a may be a container of one or more graphs 28.A graph 28 can contain other graphs28 but all run in one lifecycle,whereas the graphs 28 contained at the blueprint 28 a level may runsimultaneously, or sequentially.

A graph 28 can be represented as a file (e.g. XML file) in its blueprint28 a form or as dynamically instantiated object code at runtime. Graphs28 may be viewed as having two lives, as a running live instance and asa description of how that instance is saved for communication,transportation, distribution or storage needs. In the live context, itwill be referred to herein as a graph 28. In the description context forreference in communication, transportation, distribution or storageneeds, it may be referred to herein as a blueprint 28 a.

A simple example file is follows:

<pinConnections>  <connection>   <sourcePath>FileSource/out</sourcePath>   <destinationPath>Buffer/in</destinationPath> </connection>  <connection>   <sourcePath>Buffer/out</sourcePath>  <destinationPath>YUV to RGB/in</destinationPath>  </connection>  ... ... </pinConnections>

A graph 28 and blueprint 28 a may contain components 24, compoundcomponents 26, and may contain other graphs 28 (compound graphs 28). Acompound graph 28 can be exported and referenced as a single component24. The system 10 is operable to reference components 24, compoundcomponents 26, graphs 28, blueprints 28 a and compound graphs 28 in thesame manner.

Referring now to FIG. 5 there is shown a block diagram of an examplegraph 28 surrounded by a blueprint 28 a (e.g. final embodiment of thegraph 28) in accordance with an example embodiment. The graph 28 is acompound graph and includes an outer graph 28 and an inner sub-graph 28,both of which contain components 24 (file source 24 f, interlacedetector 24 g, film removal 24 h, deinterlacer 24 i, noise reduction 24j, file sink 24 x). The components 24 and graphs 28 may be connected bypins.

A graph 28 and blueprint 28 a may be used by system 10 to develop anddeploy a media application, and may be loaded from the repository 32 atapplication runtime. A graph 28 and blueprint 28 a can simultaneouslyhandle source data in one or more of its possible forms in a component24.

As shown in FIG. 1A, components 24, compound components 26, graphs 28,compound graphs 28, and data containers 56 are maintained in one or morelinked repositories 32. A graph 28 may implement a variety of processingroles such as an installer, manager and executor.

A component's 24 lifecycle is connected to that of its parent graph 28.Different component methods may be called during different graph 28lifecycle states. Before a graph 28 starts, its components 24 may beinstantiated and connections may be made between them. Components 24 cancomplete their configuration after they have been instantiated andbefore the graph 28 starts. When a graph 28 is loaded from file, itscomponents 24 are instantiated as soon as the graph 28 is fully loaded.Additional graph 28 and component 24 configurations may take place afterthe graph 28 is loaded, but before it starts. Once a graph 28 starts,its lifecycle may go through a realize, pre-process 1, pre-process 2,sources start and sources stop states, for example.

If the graph or one of its components encounters an error, the graph mayabort.

When a component 24 has completed its processing, it may move from theactive state to the inactive state. A graph's 28 lifecycle is done whennone of its components 24 remain in the active state. The graph 28 maybe able to keep track of the state of its components 24 unless thesecomponents 28 start their own worker threads. If a component 24 startsits own worker thread, it is responsible for setting its own activeflag. A set active method may be used for this purpose, for example.Once all components 24 have become inactive, the graph 28 may enter theFinish state.

Component 24 lifecycle actions include, for example, realize (componentsload native libraries and perform self-setup, such as allocating memoryand reading properties), pre-process 1 and pre-process 2 (componentssend their output data type information through the graph, and anycomponents that need data type information block until they receive it),sources start (source components start transmitting data, componentsprocess data coming through their input pins), sources stop (sourcecomponents stop transmitting data and processing continues until alldata has passed through the graph), abort (a signal is sent to allcomponents to cease activity and pass the abort signal to their threads,and threads may exit their run loop as soon as possible), and finish(all components are inactive and all data transmission and processinghas stopped).

If a component 24 needs to perform lifecycle-related actions, they mayneed to implement the appropriate lifecycle method or function. Examplecomponent life cycle methods include post initialize, post load fromdocument, life cycle realize, life cycle pre-process 1, life cyclepre-process 2, life cycle sources start, life cycle sources stop,process, life cycle abort, and life cycle finish.

Post initialize may be called after the component 24 has beeninstantiated, while the graph is still in the initial state. It may becalled to complete the configuration of the component by adding elementsthat can only be added once the component is initialized. Postinitialize may be implemented to create a complex data type restriction,add a property change listener, set the data type on an output pin,dynamically add pins to the component, or perform property validation,for example.

Post load from document may be called after a saved graph has finishedloading and while the graph is still in the initial state. Post loadfrom document may be implemented to configure a component based on itsconnections to other components in the graph, for example.

Life cycle realize may be the first method called when the graph isstarted. There may be no data passing through the graph when life cyclerealize is called, so the component may only have access to its ownproperties and to data types. Life cycle realize may be implemented tocreate a worker thread (worker thread is started in sources start), orget and/or verify properties that are set when the graph starts, forexample. If a property is only read during the realize state, anychanges made to the property value while the graph is running may not bepicked up or used by the component. The changes may be picked up andused in subsequent executions of the graph.

Life cycle pre-process l may be called once the graph is started and allcomponents are instantiated and configured. Empty data containers,consisting of only their data types, may be sent through the graph toprime components with run-time data type information, such as imagesizes and video frame rates. Source components implement life cyclepre-process l to send their empty data containers through the graph.Life cycle pre-process l may be implemented to provide run-time datatype information to other components. With regards to life cyclepre-process 2, as data type information is passed through the graph toprime components, the components that use the data type information canblock in life cycle pre-process 2until they receive the information theyneed to perform their configurations. Life cycle pre-process 2 may beimplemented to block until your component receives data type informationneeded to complete its configuration or perform its processing, or blocksending data through the graph until the graph is completely configured,for example.

For life cycle sources start, once the graph has been primed and all thecomponents are configured, data can begin flowing through the graph.Components can start pushing data through the graph in life cyclesources start. Life cycle sources start may be implemented to transmitdata through the graph, or start a worker thread, for example.

If source data running through the graph is stopped through externalmethods (timed broadcast, user-controlled streaming), life cycle sourcesstops may be called when a signal to stop the source is detected. Anysource clean-up (stopping threads, closing files, etc.) should beimplemented in this method. Life cycle sources stops may be implementedto stop the source stream based on an external event, for example.

The process method may be called any time data arrives on a component'sinput pin. The process method is where all data processing occurs. Theprocess method is where data is retrieved from input pins and placedonto output pins. The process method may be implemented to transformdata, retrieve data from input pins, push data onto output pins, changeor set properties, and so on.

If an error occurs in the graph, life cycle abort may be called. Lifecycle abort is used to stop threads and close files. Life cycle abortmay be implemented to stop worker threads, or close files, for example.

Life cycle finish may be the final method called in the graph lifecycle.It may be called when no components remain in the active state. Anyfinal clean-up needed to be done should be implemented in this method.Life cycle finish may be implemented to close files, releaseallocations, close socket connections, or wait for internal threads tofinish, for example.

The repository 32 is operable to manage versioning of components 24,graphs 28, and blueprints 28 a in order to keep track of updates,variations, and modifications made to components 24, graphs 28, andblueprints 28 a. The repository 32 is operable to handle runtimelibraries and engines used by graphs 28, blueprints 28 a, and components24, such that the repository is self-managed with respect to versioning.The repository 32 is further operable to receive the developmentframework 12 to manage versioning of and updates to the developmentframework 12. That is, the repository 32 can load up-to-date versions ofthe development framework 12, including runtime libraries and engines.The development framework 12 may be loaded upon request so thatappropriate and updated versions of the development framework 12 areused. The graphs 28 and blueprints 28 a are loaded at run time so thatthe appropriate version of the graph 28 and each component 24 in thegraph 28 is used. A blueprint 28 a may reference a solution set ofcomponents. A solution set of components is a set of components 24 andspecific versions of each component 24 in the set. The blueprint 28 amay reference a solution set of components using a label. A blueprint 28a may reference a solution set using a label in order to dereferencefrom the specific components 24 and versions of the solution set. Thatway, if the solution set changes, such as if a component 24 is added orremoved from the solution set, or a version of a component 24 changes inthe solution set, then the same label will reference the updatedsolution set without requiring modification to the blueprint 28 acontaining the label. This may result in more efficient processing as areduced number of modifications and updates are required. Further,components 24 may be self-contained and isolated from a dependency pointof view. The entire dependency set of a component 24 may beself-contained, being specified and packaged in the componentdistribution unit (e.g. plugin). The component 24 dependencies may alsobe isolated, referring exclusively to the specific component 24 andversion(s) they depend on. This may enable the system 10 to realizecomplex workflows while resolving components 24 dependencies withoutuser intervention. Further, the dependency isolation may allow thesystem 10 to provide distinct behavior while executing blueprints builtwith the same components 24 by isolating the different versions of thesecomponents 24 and their dependencies. Processing errors may also bereduced as the system 10 and user may not have to manually track andmanually update components defined by blueprints 28 a or graphs when alabel is used.

Referring now to FIG. 13 there is shown a block diagram of an exampleinterface 100 for defining a solution set of components in accordancewith example embodiments. In this example, the interface 100 displaysdifferent types of components 102 along one axis and different versions104 of each type of component along another axis. In this example thereare 6 different types of components 102 and each type may be associatedwith a component identifier such as for example, c1, c2, c3, c4, c5, c6.System 10 may use the component identifier to reference a particulartype of component.

There may be multiple versions 104 of each type of component, or sometypes of components may only have one version. For example, a type ofcomponent 102 c1 may have 8 different versions. Each version may beassociated with a version identifier such as for example: c1v1, c1v2,c1v3, c1v4, c1v5, c1v6, c1v7, c1v8. System 10 may use the versionidentifier to reference a particular version of a specific type ofcomponent.

A solution set 106 references a set of components, and more particularlymay reference a specific version of each type of component for use by acomputing application. In this example, the solution set 106 is thespecific version of each type of component that is intersected by a line(c1v4, c2v3, c3v4, c4v1, c5v2, c6v1). The solution set 106 may bemodified by changing a version of a specific component, by removing aparticular type of component, by adding a new type of component, and soon. The interface 10 may provide a mechanism for a user to efficientlymodify, test, and deploy different versions of components by changingthe solution set. For example, the interface 100 may change a solutionset 106 by sliding a new version of a component to intersect with theline, sliding all versions of a type of component line so that noversions of a particular type of component intersect with the line (i.e.which may indicate that a particular component is no longer part of thesolution set), and so on. System 10 is operable to test and deploy amodified solution set 106 for a particular computing application, andcan also test all updates made to a solution set 106.

A blueprint 28 a is operable to reference a particular solution set 106using a label to dereference from the specific components and versionsof the solution set. If the contents of a solution set 106 changes thenthe blueprint 28 a label will reference the changed solution set 106without requiring modification to the blueprint 28 a. Multipleblueprints 28 a may reference the same solution set 106. A blueprint 28a may also reference multiple solution sets 106. If a change is made tothe solution set 106 and a label is not used to dereference from thespecific components and versions of the solution set, then multipleblueprints 28 a may require updating and tracking as referencing thesolution set 106 in order to ensure that the blueprints 28 a referencethe appropriate components 102 and versions 104 thereof.

Also, in accordance with some embodiments, a solution set may itselfhave a version number. The version number can be referenced by the labelof the blueprint 28 a to identify which version of the solution set theblueprint is working with. In other embodiments, a blueprint may ignorethe version number and automatically update to refer to the latestversion of the solution set. The solution set version number may providea mechanism to maintain and create a history of solution sets as changesare made thereto.

The development framework 12 enables complexity abstraction by definingan entire graph 28 or blueprints 28 a as a component 24. A component 24which defines an entire graph 28 may in turn be used in another graph 28or blueprint 28 a, such that a graph 28 may be embedded within anothergraph 28 or blueprint 28 a. The graph 28 or blueprint 28 a may referencethe component 24 using a label to dereference from the specific instanceof the component 24. That way, if the component 24 is modified then thelabel of the graph 28 or blueprint 28 a will reference the modifiedcomponent 24 without requiring additional modification to the graph 28or blueprint 28 a. That is, the graph 28 or blueprint 28 willautomatically update to reference the modified component 24 by virtue ofthe label reference. The development framework 12 also providesproperties redirection through the ability to expose a component 24property as a property of an enclosing graph 28. The developmentframework 12 further enables modular and recursive construction ofgraphs 28 through the use of pre-packaged graphs 28 and components 24.

Commands, along with events, provide a means for components, graphs andcalling applications to communicate outside of the regular data flow.Commands are functions that may be called on components. A command maybe executed by passing the command name and an argument (in the form ofa data container 56) to a process command method. A data container 56may be passed to process command method, even if the command will not beusing the data. If the command will not be using any data, an empty datacontainer 56 may be passed to process command function.

Command pins may act as triggers to call commands on components. When adata container 56 is placed on a command pin, the process commandfunction may be called with the command name (the name of the pin) andthe data container 56 as arguments. The data in the data container 56may be used by the command, but it does not need to be. Commands may bedefined within the component definition file, such as a plugin.xml filefor example. Commands may have attributes or properties, such as a name(which may be used to access the command) and description (which mayinclude information such as what the command does and any data thecommand expects including data type. An example command may be “read”(name) which reads a certain number of bytes starting from a particularlocation and takes a data container with the number of bytes and a seekposition (description).

Events, along with commands, may provide a means for components, graphsand their calling applications to communicate outside of the regulardata flow. Events may follow an event listener pattern where componentsfire events to registered event listeners implementing a node eventlistener interface. A node event may include the event name (String) andraw data (Object). The identity of the component that fired the eventmay also be contained within a node event.

Events may be exposed on pins. When an event occurs, the data objectgenerated by the event may be encapsulated in a data container that ispushed onto the event pin. If the event generates a null object, anempty data container may be placed on the pin. The propagation of a datacontainer on an event pin may signal that the event has occurred. Eventsmay be defined within a component's definition file, such as for examplea plugin.xml file. Events may have attributes or properties, such as aname (which may be used to access the event) and description (which mayinclude information such as under what circumstances the event is firedand whether the event contains any data

Capabilities may be externally defined contracts that define how acomponent with a particular capability should behave and appear. Acapability definition may include information such as the properties andpins that are expected in a component with this capability. Capabilitiesare intended for components, graphs and applications to requestcomponents dynamically based on their functionality. A component maydeclare that it implements more than one capability. Capabilities aredeclared in the component definition file, such as for example in aplugin.xml file. Capability names may be unique.

Data processing in a component can follow either a push or pullparadigm. In the push paradigm, data is pushed onto a component's inputpin, calling that component's process method. In the pull paradigm, acomponent will request data from its input pin on a separate internalthread. The internal pulling thread will block until data is received.

When a graph is started, source components may send a priming datacontainer consisting only of data type information through their outputpins. The data type information in these “empty” containers may be usedby components downstream to complete their configuration. A sourcecomponent will initially produce an empty data container. Once the emptydata container has been sent through the graph, real data will beoutput. All components should check for empty data containers arrivingon their input pins. When an empty data container is received on aninput pin, the component should validate the data type information andsend out an empty data container of its own consisting of its outputdata type. An application programming interface class may provideconvenience methods for pushing containers, including empty containersonto output pins.

Passing mutable data containers may allow components to perform dataprocessing without making copies of the data. However, in-placemodifications should only be done if the data is not being usedelsewhere. If the data will be used in two locations (threads,components, methods, etc.) at once, it may be made immutable. If anoutput pin feeds into multiple input pins, the same data container willbe passed to each input pin; the data container will automaticallybecome immutable to prevent the receiving components from modifying thesame object.

A component may call clone if immutable method on the input datacontainer if it will be modifying the data container's data object aswell as its data type. The method clone if immutable returns a clone ofthe data container if the data container is immutable; otherwise itreturns the data container itself. The method clone if immutable willmake a full copy of the entire data object in the data container,potentially using a substantial amount of memory. If the component onlyneeds to modify the data type within the data container, then clone ifimmutable should only be called on the data type before it is modified.

All stream data type definitions may inherit from a base stream datatype definition (data type stream). The stream data type definitionincludes an end of stream key that indicates whether or not this datacontainer is the last. Marking end of stream is important to signal thatno other data will be arriving for processing. All stream sourcecomponents may set end of stream to true on the last data container theysend. The end of stream data container can be empty (no data object).Stream processing components may check for end of stream in the dataType of each data container they receive. When they receive the end ofstream data container, they must in turn send out an end of stream datacontainer.

In the push data processing model, data gets pushed onto the component'sinput pin, calling that component's process method. The process methodis effectively called by the component pushing the data onto the inputpin. The push model is a passive model. Data containers are “pushed”onto the input pin by the previous component in the workflow. A processmethod may be called whenever a data container arrives on the input pin.The component may not be active unless it is processing data.

The push model may be used in cases where there is only one pin, or incases where the input of multiple pins do not need to be coordinated. Ifthe data containers arriving on multiple input pins need to becoordinated, then the pull model may be used.

In the pull model, the component will block until either a datacontainer arrives on the input pull pin or a timeout expires. A workerthread may be used to drive pulling data from the input pins, processingthe data, and pushing output to the next component. Methods may becalled on pull input pins to pull the data, blocking until either a datacontainer arrives on the pin, or a timeout expires. If the timeoutexpires before a data container arrives on the pin, a null datacontainer may be returned.

To prevent a pull component from entering a busy loop, it is preferableto block until data arrives rather than until a timeout expires. Howeverthere are cases where a timeout is necessary, for example if one pinonly receives sporadic input. If one pin will not always receive data,the component can use a timeout on this pin to allow it to continue itsprocessing without this pin's input. Unlike a push processing component,a pull processing component needs to be aware of when to stopprocessing. An end of stream data container can be used for thispurpose. If the parent graph is aborted, a null data container will bereturned by the pull method. Pull components may need to verify whethertheir parent graph has aborted whenever they receive a null datacontainer.

Visual Design Subsystem

The visual design subsystem 30 is operable to output graphs 28 andblueprints 28 a for developing media applications using components 24,compound components 26, blueprints 28 a and other graphs 28. The visualdesign subsystem 30 defines relationships between components 24,compound components 26, and graphs 28 using pins to define theconnections.

In one example embodiment, the visual design subsystem 30 may beaccessible via a cloud computing system. The visual design subsystem 30may allow a user to create components 24 and graphs 28, and define anorder of operations or workflow for the graph 28. The visual designsubsystem 30 may allow the user to group components 24 (includingcompound components 26 and graphs 28) into functional blocks and arrangethose functional blocks into specific orders of operation. The visualdesign subsystem 30 further allows the construction of logic brancheswhich allow for flexibility in execution of the components 24 orfunctional blocks. The visual design subsystem 30 may also allow for theconstruction of functional blocks which may operate linearly in time,non-linearly, or as discrete operations with separate lifecyclemanagement.

The visual design subsystem 30 defines a graph by connecting components24, compound components 26, and graphs 28 using connection mechanismssuch as pins.

The visual design subsystem 30 allows parameters for the components 24to be set and monitored. The visual design subsystem 30 may also allowgraphs 28 to be instantly reviewed to test functionality andperformance. The visual design subsystem 30 may simplify component 24and graph 28 testing, development and deployment.

The visual design subsystem 30 may provide an interface, such asinterface 10 of FIG. 13, in order to define solution sets of components24 and versions thereof for use in graphs 28, blueprints 28 a, and othercomponents 24. The visual design subsystem 30 is operable to test anddeploy a solution set for use in graphs 28 and blueprints 28 a. Thevisual design subsystem 30 is operable to test and deploy a version of acomponent 24 for use in a solution set for graphs 28 and blueprints 28a.

Referring now to FIG. 6 there is shown a block diagram of an exampleinterface for a visual design subsystem 30 in accordance with an exampleembodiment. The example interface for a visual design subsystem 30includes a graph 28 and components 24 (file input 24 u, color spaceconverter 24 v, logic branch 25 with routing based on image width andheight, scaler 24 w and AVC encoder 24 y). Other example components 24include YUV to RGB, java image controller, scripted component, flowcontrol component, and so on. The example interface for a visual designsubsystem 30 illustrates an interface 80 for setting the properties 25and values 27 for components 24.

The visual design subsystem 30 outputs a graph 28 or blueprint 28 a,which may be stored in the repository 32 for subsequent use andreference. For example, the visual design subsystem 30 may output a file(e.g. XML file) which describes a graph 28, components 24, compoundcomponents 26, blueprints 28 a, and compound graphs 28. The filedescribes the components 24 that are used, the parameter values, and theconnections between the pins of the components 24. A graph 28 may beused as part of media applications and may be loaded by the system 10 atrun time to ensure the appropriate components 24 of the graph 28 areused. For example, a graph 28 may reference a solution set of components24 and versions thereof. The solution set may change or update, andbecause the graph 28 references the solution set by label theappropriate solution will be loaded by the system 10 at run time. Therepository 32 maintains a collection of graphs 28, blueprints 28 a, andcomponents 24. The repository 32 manages versioning of components 24 andgraphs 28 to keep track of updates made to components 24 and graphs 28,and new versions thereof. The graphs 28 are loaded at run time so thatthe appropriate version of the graph 28 and each component 24 in thegraph 28, as defined by a solution set for example, is used. Further,the graphs 28 may reference the solution set by label so that if thesolution set is changed the graph 28 will automatically reference thechanged solution set without requiring a manual update to the graph 28.That is, the blueprint with the label may automatically reference thechanged solution set without requiring a manual update. A solution setmay be referenced by different blueprints 28 a using the same ordifferent labels. For example, a user may configure a blueprint 28 awith a label for a solution set, such as “ready for testing” or “passedtesting” and another user may configure the same or different blueprint28 a with a different label for the same solution set, such as “MY SET”,for example. The label provides a descriptive mechanism for a user andalso provides efficient processing and propagation of updates. The labelmay continue to reference a solution set even if a modification is madethereto. Labels may also be used to reference components 24, blueprints28 a, graphs 28, and so on. Different labels may be used to referencethe same components 24, blueprints 28 a, graphs 28, and so on.

The visual design subsystem 30 may export a blueprint 28 a or a graph28. For example, the blueprint 28 a may be instantiated on a desktopplatform as a local engine or subset of an application, or in the cloudby a cloud engine 36. A blueprint 28 a may be considered to be a finalembodiment of a graph 28. A blueprint 28 a and a graph 28 reference asolution set of components and versions thereof using a label.

The visual design subsystem 30 may be an engine or object code that canbe run through an application interface or through the set of SDKs. Thevisual design subsystem 30 is operable to construct graphs 28, testgraphs 28, perform run time validation, and simulate graphs 28.

The visual design subsystem 30 may perform design time media inspectionand propagate media type, data container information and componentconfiguration changes across graphs 28 and blueprints 28 a thusvalidating proper realization of the graph 28 and blueprint 28 a into amedia application that can process the desired type of media. Forexample, labels may be used to reference solution sets so that if thesolution set changes then label used in the blueprints 28 a will alsoreference the updated solution set without requiring the blueprint 28 ato be updated. The visual design subsystem 30 enables complexityabstraction by defining an entire graph 28 or blueprint 28 a as acomponent 24. Accordingly, data containers 56, components 24, compoundcomponents 26, graphs 28, and blueprints 28 a may be generally referredto herein as components 24, and may be used like components 24 asbuilding blocks for computing applications.

The visual design subsystem 30 may provide properties redirectionthrough the ability to expose a component 24 property as a property ofan enclosing graph 28. The visual design subsystem 30 enables modularand recursive construction of graphs 28 through the use of pre-packagedor pre-constructed graphs 28 and components 24. The visual designsubsystem 30 uses the repository 32 to provide graph 28 and blueprint 28a persistence storage and versioning strategy enabling backwardcompatible changes. The visual design subsystem 30 provides dynamic,override-able and decoupled user interface support.

Referring now to FIG. 1B there is shown a block diagram of the data flowof a system 12 for dynamic development and deployment of computingapplications, in accordance with an example embodiment.

The system 12 may include a user system 14 delivering a plug in packagethat may contain one or more components 24, graphs 28, and blueprints 28a. The system 12 is also shown to include a repository 32, agent 34,engine 36 and an application system 15.

Components 24 may be stored in a repository server 32. The repositoryserver 32 manages the components availability, versioning andOS/platform capability. When a new job is running the agent 34 willcontact the repository server 32 to acquire the components 24 requiredby the engine 36 which will be running the graph 28/blueprint 28 a.

Components 24 may be delivered to the repository server 32 as PluginPackages. The Plugin Packages contain one or more components 24 ofrelated functionality. The Plugin Packages may also include graphs 28 orblueprints 28 a for example. Note that each Plugin Package may also besigned and have a manufacturer's digital certificate. Third party PluginPackages may require a certificate with their company's identifierbefore the package may be recognized by the repository 32. Thiscertificate may be provided by a certification agent, as will bedescribed in relation to FIG. 14.

The visual designer 30 may provide a graphical interface that can beused to create new graphs 28, or to create new compound components 26based on existing components 24. Compound components 26 includecomponents 24 embedded within other components 26, and may be referredto herein simply as components 24. Components 24 are the basic dataprocessing elements. Components 24 may have input pins (which allow datato enter the component), output pins (which allow data to leave thecomponent) and settings (which allow the user to set some of theparameters/properties which define what happens to the data when it isprocessed by the component). Compound components 26 can be created usingexisting components 24 and these compound components 26 can be saved asnew components 24.

A graph 28 is a set of connected components 24. Components 24 areconnected via their pins. Data is encapsulated and passed betweencomponents in data containers 56. A data container 56 may be comprisedof a data object (the raw data that is being processed) and a data type(meta-information about the raw data).

A graph 28 can be a specific workflow solution or a graph 28 can beembedded within another graph 28 as part of a more complex workflow.Complete workflow solutions can be saved to the repository 32 asblueprints 28 a.

Deployment Subsystem

The deployment subsystem 14 may include one or more linked repositories32, a license server 42, cloud agents 34 on user computing systems,cloud engines 36 run by the cloud agents 34, a job manager 50, and asecurity module 46. The deployment subsystem 14 provides externalinterfaces 38 to repositories 32 to manage components 24, blueprints 28a and graphs 28, to the job manager 50 to manage application jobs, andto cloud engines 36 to manage the execution of graphs 28 and blueprints28 a.

The deployment subsystem 14 may include a computing application used tomanage the workflow and to define the graphs 28. This application mayoptionally provide access to a graph creation tool such as the visualdesigner 30. The deployment subsystem 14 may include an agent 34 whichmay exchange commands and status between the application and engines 36.One agent 34 can communicate with more than one engine 36. Thedeployment subsystem 14 may include an engine 36 which is operable forrunning components 24 in a graph 28.

The deployment subsystem 14 may include a license server 42 used byengines 36 to check in and out licenses for the purchased components 24.The license server 42 may also be used to enable the application. Thedeployment subsystem 14 may include a repository server 32 used to storethe components 24 that are to be deployed on engines 36.

There may be two types of deployments: stand-alone/desktop deployment;and network deployment.

Referring now to FIG. 18 there is shown a block diagram of stand-alonedeployment. In this type of deployment the application 47 accesses thedevelopment framework 12 API directly. All of the components of thedeployment can be installed on a single host system 49. That is, thelocal disk is used to store components 24, graphs 28 and blueprints 28a. Alternatively, the repository 32 may be used instead of the localdisk to provide a database of plugin packages. The repository 32 can beused by more than one host system 49. The license server 42 can beinstalled on the host system 49 for a true “stand alone” deployment, orit can be installed on the network so that it can be accessed by morethan one host system 49, to allow for network licensing.

Referring now to FIG. 19 there is shown a block diagram of networkdeployment. In this type of deployment an agent 34 is required tocommunicate with the higher level management application 55 and tocommunicate with the engines 34. The agent 34 may reside on one to ndifferent host systems 51, 53. Access to a repository 32 may be requiredfor network deployment.

An agent 34 may be the dispatch/coordinating service installed on allhost systems which will run engines 36. An agent 34 may coordinatemanagement and monitoring of systems on the network and dispatches andmonitors jobs running on engines 36. An agent 34 communicates withhigher level applications (for example, job manager 50) through a webservices interface, for example. Agents 34 may include a communicationservice and a server service. Agents 34 can coordinate management andmonitoring of more than one engine 36 or a mix of engines 34 on the samesystem, the only restriction may be the practical limits of the hostsystem's resources (cpu, memory, bandwidth, etc).

An engine 36 is a running version of a graph 28 or blueprint 28 a. Anengine 36 may access source files, write output files and return statusto the agent 34 or Kayak-based application. An engine 36 communicateswith the agent 34 to acquire the required components 24 and with thelicense server 42 to authorize the components 24 required to run thegraph 28.

The repository 32 stores components 24, compound components 26,blueprints 28 a and graphs 28. As one example, the repository 32 may bea web services based repository accessible through a cloud computingsystem via external interfaces 38. As another example, the repository 32may be stored on a local system. The deployment subsystem 14 may use oneor more linked repositories 32 for version management, maintenance anddeployment. The repository 32 is a hosted collection of components 24and graphs 28 which are accessed by a protocol, identified as required,and transferred to the target host environment. As an illustrativeanalogy, a graph may be viewed as a recipe (i.e. template) listingdifferent ingredients (i.e. components) and the repository 32 containsthe blueprint 28 a for the graph 28 and components thereof to providethe user with both the “recipe” and the “ingredients” listed in the“recipe”.

The repository 32 organizes each component 24 (regardless of whether itis a standalone component 24 or is a compound component 26, graph 28,blueprint 28 a, or solution set with reference to other components 24)with respect to revision (i.e. versions of the component), ownershipstructure, licensing requirements, and dependencies on other components24 or technologies. These dependencies or requirements may furtherrequire specific revisions of technologies or components 24 for properfunction.

The repository 32 manages versioning such that it can determine the mostappropriate version of a component 24, graph 28, and blueprint 28 a. Theappropriate version of a component 24 may be defined by a solution set.The repository 32 allows access to any of the available versions ofcomponents 24 and graphs 28, which may include the most recent versionbut necessarily. The repository 32 may interact with interface 10 inorder to provide available versions for each component and definesolution sets. For example, a customer may want a version of a component24 that they have tested instead of the latest version, and may includethe tested version in the solution set. This is may be important fordownstream management. When a graph 28 and blueprint 28 thereof is usedby an application the components 24 defined by the solution setreferenced by the label in the blueprint 28 a or graph 28 are loadedfrom the repository 32 at media application runtime so that the properversion of the components and graphs are used. The repository 32 isconfigured to provide versioned components 24 with multi stagecapability, automatic component update propagation and gated componentupdate release.

Referring now to FIG. 7 there is shown a block diagram of an exampleuser interface 90 for a repository 32 in accordance with an exampleembodiment. The example interface 90 for the repository 32 displays anaddress 91 for the repository, such as a uniform resource locator. Theexample interface 90 for the repository 32 displays a listing 92 ofnames 93 of components 24, compound components 26, and graphs 28, alongwith an associated description 96, provider 95, and version 94. Thelisting 92 may also include an associated status, such as complete,tested, and so on. The interface 90 may also include the interface 10 ofFIG. 13.

There may be multiple linked repositories 32 a, 32 b and a mediaapplication can access the multiple repositories when a graph 28 orblueprint 28 a is used by the media application at runtime. Examples ofrepositories 32 include staging, preproduction, and production.

A cloud agent 34 may be provided to a user computing system to managethe local resources of the host computing system. The term ‘cloud’ asused herein may describe a heterogenous environment where agents canlive in the cloud or on desktops, laptops, mobile devices, and so on,and is not limited to ‘cloud computing systems’ accessible through theInternet. That is, a cloud agent 34 may also refer to a desktop agent,local agent, and so on. The cloud agents 34 may interact with cloudengines 36 to execute graphs 28 and blueprints 28 a thereof in order torun media applications, or other computing applications. At applicationruntime, a pool of one or more cloud agents 34 can access a sharedrepository 32 of components 24 and graphs 28 to construct theapplication. A cloud agent 34 is operable to instantiate blueprints 28 aof a graph 28 and run them in a cloud engine 36.

A cloud engine 36 provides a running environment for blueprints 28 a ofgraphs 28 and creates media applications on the blueprints 28 a of thegraph 28. The term ‘cloud’ as used herein may describe a heterogenousenvironment where engines can live in the cloud or on desktops, laptops,mobile devices, and so on, and is not limited to ‘cloud computingsystems’ accessible through the Internet. That is, a cloud engine 36 mayalso refer to a desktop engine, local engine, and so on. The cloudengine 36 is a runtime construct which receives blueprints 28 a ofgraphs 28, analyzes and organizes component 24 dependencies, executesprotocols for retrieval of the required components 24, constructs thosecomponents 24 into new run-time executables and dispatches thoseexecutables against a dynamic job or process. The dispatch of newrun-time executables can be persistent or dynamic in nature. Inpersistent mode the cloud agent 34 registers the availability of cloudengines 36 with the calling server or application and no furtherdeployment (installation) is required. In dynamic mode each executablecan be ‘renewed’ at each job instantiation creating a new ‘product’ witheach deployment.

The cloud agent 34 can be implemented as a desktop application or acloud based application (using an external interface 38). For a cloudbased application, the cloud agent 34 may be required to manage thecloud engine 36 and provisioning for specific components 24, graphs 28,blueprints 28 a and other resources. For the desktop application, adynamic link library may be used and the system SDK 20 may allow fordynamic updates of components 24 and graphs 28.

The cloud engine 36 is operable to coordinate the license server 42 andthe repository 32. The cloud agent 34 is operable to dispatch, manage,and run independent, unrelated functionality on a single host system.

The cloud agent 34 is operable to provide interfaces and control overlifecycle of functional blocks of components.

The cloud agent 34 is operable to monitor, aggregate, and reportinformation about the environment in which it is running to allow formaximum optimization and balance of work.

A cloud engine 36 is operable to execute a graph 28 or blueprint 28 athereof at application runtime in order to construct and deploy a mediaapplication, or other computing application. At application runtime acloud engine 36 is operable to use a blueprint 28 a of a graph 28 andthe solution set referenced in the blueprint 28 a in order to identifycomponents 24 and other graphs 28. Further as a facilitator for versionresolution, components 24 may be self-contained and isolated from adependency point of view. The entire dependency set of a component 24may be self-contained, being specified and packaged in the componentdistribution unit (e.g. plugin). The component 24 dependencies may alsobe isolated, referring exclusively to the specific component 24 andversion(s) they depend on. This may enable the system 10 to realizecomplex workflows while resolving components 24 dependencies withoutuser intervention. Further, the dependency isolation may allow thesystem 10 to provide distinct behavior while executing blueprints builtwith the same components 24 by isolating the different versions of thesecomponents 24 and their dependencies.

The cloud engine 36 is operable to send a request to the repository 32for the identified components 24 and graphs 28, receive a copy of thecomponents 24 and graphs 28 from the repository 32, and dynamicallybuild a media application using the components 24 and graphs 28. Cloudagents 34 run the cloud engines 36. A cloud agent 34 is operable toinstantiate blueprints 28 a of graphs 28 and run them in a cloud engine36.

A cloud engine 36 is registered with a shared repository 32 anddispatched by job manager 48. The shared repository 32 works similar toa local repository but its contents are shared by a pool of cloud agents34. The job manager 50 dispatches blueprints 28 a of graphs 28 cloudagents 34 referencing available licenses in the license pool 44 asmaintained by the license server 42.

The cloud agent 34 may provide life-cycle management services for thecloud engine 36 which in turn manages the components 24, blueprints 28 aand graphs 28. The cloud engine 36 is operable to control all componentsin a multi-threaded and multi-process execution environment and tomanage initialization. The cloud engine 36 may enable early propagationof data type information. The cloud engine 36 may provide graceful andnon-graceful termination.

The cloud engine 36 is operable to provide component configurationservices for graph 28 execution. The cloud engine 36 is operable toprovide the ability to auto-configure component 24 settings based on theinput data type avoiding unnecessary user input.

The cloud engine 36 is operable to provide the ability to configureindividually each input pin to function according to a push or pullmodel allowing heterogeneous components 24 to connect to realize thegraphs (blueprints).

The cloud engine 36 is operable to provide memory management servicesthrough memory pools, garbage collection and lifecycle management forlarge data objects.

The cloud engine 36 is operable to manage data communication pathways inbetween components 24 allowing them to connect and pass data to realizethe blueprints 28 a.

The cloud engine 36 is operable to define generic media data type andmetadata model (video, audio, time code, subtitles, closed captions), aspecific application domain data dictionary, a mechanism to encapsulatedata and data-type information with data packets for richer informationand optimizes data container management The cloud engine 36 is operableto provide hierarchical data-type representation of the informationoccurring in the graph. The cloud engine 36 is operable to providedata-type transformation strategies to ease component manipulation ofdata-types.

The cloud engine 36 is operable to provide multithreaded data integritythrough immutable (read-only) packets and data access performanceoptimization, components altering ‘writable’ packets in-place, copyingonly read-only data.

The cloud engine 36 is operable to provide out of process executionsupport, thus enabling blueprints execution in separate processes, whilemanaging large data structures transfer, inter process communication andtransparent shared memory when possible.

The cloud engine 36 is operable to provide support for multi-languagecomponent development with communication and interoperability betweenthem.

The cloud engine 36 is operable to provide cross platform applicationexecution support, allowing graphs to be executed on multiple types ofplatforms, including Windows, Mac, Linux platforms, for example.

The license server 42 is operable to dynamically manage a license pool44 of licenses and associate licenses with components 24 and graphs 28.The license server 42 is operable to determine whether a requesting userhas the appropriate license for the components 24 identified in a graph28 that forms part of the media application. A user may only bepermitted to use components 24 and graphs 28 if they have the requiredand appropriate license. This allows a user to use the technology acrossdepartments, groups and companies depending on the conditions of thelicense associated with the various components 24 of the graphs 28.Further, this enables a provider to control and track use of itscomponents 24 and graphs 28. The license server 42 provides tracking ofall ‘in use’ technology and provides for a central accounting mechanism.The licenses can be controlled by concurrency, physical system,floating, and leased.

That is, the license server 44 provides runtime authorization tocomponents 24 through a pool of available licenses. Referring now toFIG. 17, there is shown an example browser based console 41 which can beused to access the license server 44 to show which features areavailable, how many are currently in use, and which systems are usingthe features. The license server 44 can access and/or import plug-inpackage 43 of licenses. Engines 36 can access the license server 44 tocheck out licenses for the components 24 required to run a graph 28 andto check in those licenses once the graph 28 has finished running. Anapplication system 45 can access the license server 44 to check outlicenses for the components 24 required to run an application and tocheck in those licenses once the application has finished running.

The job manager 50 is configured to provide job/engine dispatch,failover, tracking and reporting. The job manager 50 dispatches cloudengines 36 based on available resources, system availability, processingcapability, available licenses in the license pool 44 maintained by thelicense server 42. In particular, the job manager 48 dispatches cloudengines 36 based on the latest or appropriate graph blueprints 28registered with production repository 32 and available licenses in thelicense pool 44. The job manager 50 may also be configured for mappinggraphs 28 to cloud engines 36. The job manager 50 may also be configuredto provide the highest level of access to the running cloud engines, andprovide centralized access to the cloud engines 36 regardless of state(running or not). The job manager 50 may further self-extend interfaces(e.g. web services) based on the graph 28/blueprint 28 a that is loadedon the cloud engine 36 to provide a namespace (for example, similar tothe web) which may allow the developer to discover which graphs 28 andcomponents 24 are used in that particular computing application, queryparameters, set parameters, and so on.

Referring now to FIG. 8 there is shown a block diagram of an exampleinterface 60 for a job manager 50 in accordance with an exampleembodiment. The interface 60 provides a listing 62 of resources managedby the job manager 50 including a start time, end time, resource name,status (e.g. running, completed, failed, cancelled), progress, averageencoding rate, estimated time remaining, failures, source file, project,and notes. The interface 60 may also include a search box 64 forsearching for jobs managed by job manager 50. The search box 64 mayprovide a variety of search parameters such as date range, project name,resource, group, current state, and events (e.g. dropped, failover), forexample. The interface 60 is operable to provide a variety of pages orwindows, such as a summary, network monitor, groups, resources,schedule, jobs, and alerts, for example.

The security module 46 provides for secure connections andcommunications within system 10.

A code signing module 40 is operable to digitally sign each component 24to associate a developer, license, or both with each component 24.

The translation module 58 is operable to translate multiple languagesinto a common language for system 10.

An interface application may provide users with a way to create graphs28 and to run those graphs 28. The graph 28 creation may beprogrammatic, where a graph 28 is generated based on a few user selectedparameters, and the actual graph 28 itself is hidden from the user. Atthe other end of the spectrum the interface application may provide fullaccess to the visual designer 30, with the user choosing and connectingthe components in the graph 28 manually. The interface application mayalso provide a way to select the data inputs for the graph 28 (e.g.,source files), to set the outputs for the graph 28 (e.g., archivefiles), and to monitor and control the execution of the graph 28.

An example of an interface application is job manager 50 with jobengines 34. The job manager 50 may be a media manager server whichmanages file transcode jobs. User access to the Media Manager Server maybe via an interface application. Jobs are submitted to the server byadding source files to watch folders. Watch folders are associated withjob projects (graphs 28 for transcoding jobs). Graphs 28 may be createdusing a customized version of the visual designer 30. The Media ManagerServer may have access to a pool of transcode host systems, and eachtranscode host system may communicate with the Media Manager Serverusing an agent 34 installed on the host. When a job is submitted thesource file and project (graph 28) are sent to a host system with anagent 34 which will then manage the engine 36 which processes the job.Status is returned to the Manager Sever while the job is being processedand when the job completes.

Referring now, to FIG. 1C there is shown a block diagram of the dataflow of a system for dynamic development and deployment of mediaapplications, in accordance with an example embodiment. A blueprint 28 ais a container of one or more graphs 28. A graph 28 can contain othergraphs 28 but all run in one lifecycle, whereas the graphs 28 containedat the blueprint 28 a level may run simultaneously, or sequentially.Cloud agents 34 and cloud engines 36 may be operable to receive ablueprint 28 a and use it to instantiate a graph 28 of components 24,compound components 26, and data containers 56.

The normalization module 52 is operable to receive input media files 54(which may be files as in the original document, live media and so on),and convert and parse the input media files 54 into data containers 56to be processed by the graph 28/blueprint 28 a. The normalization module52 extracts as much data as possible from the input media file 54 topopulate the data containers 56 and the data type and data objects ofthe data containers 56. The normalization module 52 can match the inputdata to a dictionary of languages linked to data types in order topopulate the data type component of the data containers 56.Normalization module 52 capability may be distributed across variouscomponents (being actually provided by specific components or forexample a media file input component).

System 10 may be implemented as a cloud computing system and the usermay access system 10 through external interfaces 28 (such as webservices for example).

Referring now to FIGS. 9 and 10 there is shown block diagrams 70, 80 ofexample web services implementations in accordance with exampleembodiments.

As shown in FIG. 9, web services 72 may connect and interact with abroadcast system 74, post broadcast system 76, and cable/IPTV system 78.Web services 72 may provide a virtual layer between external systems(e.g. service system 74, post processing system 76, cable/IPTV system78) and the components of system 10. The web services 72 interact withjob manager 50, which in turn dispatches and manages one or more cloudengines 36. The job manager 50 may also interact with license server 42and license pool 44 to comply with license restrictions. The webservices 72 may be provided as a cloud computing system. One feature ofembodiments described herein is automatic generation web services 72based on the components that exist in a running engine. The web services72 can be further filtered through access control by the author/designerof the application.

As shown in FIG. 10, web user interfaces 81 may connect and interactwith one or more service system(s) 86, post processing system(s) 85, andother external systems 87 (e.g. cable/IPTV system). Web user interfaces81 may provide a virtual layer between external systems (e.g. servicesystem(s) 86, post processing system(s) 85, other system(s) 87) and thecomponents of system 10 (referred to as virtual appliances 84). In someembodiments, some web user interfaces 81 may interact with anapplication 82 a to access the components of system 10. In someembodiments, business logic residing on web servers 83 is operable tocontrol interactions between web user interfaces 81 and the componentsof system 10.

An example embodiment may implement an asset management and publishingsystem that is responsible for implementing actions including storing,conforming, searching, and publishing large amounts of media data toindividual publishing profiles. A publishing profile can be a VODtarget, a device, a service, and a network, for example. The assetmanagement and publishing may be implemented as a web application.

Referring now to FIGS. 11 and 12 there is shown diagrams of example dataflows for implementing an asset management and publishing system inaccordance with example embodiments. At 200, system 10 is operable toingest digital media assets (such as input media files). At 202, system10 is operable to determine whether the digital media assets meetcodified requirements and use job manager 50 to dispatch cloud engines36 for processing and modifying the digital media assets to meet suchcodified requirements. At 204, system 10 is operable to store digitalmedia assets. In some embodiments, the job manager 50 may be used tomanage the storing of the digital media assets. At 206, system 10 isoperable to search through the digital media assets for refinement basedon search parameters. At 208, system 10 is operable to publish theprocessed and refined digital media assets to a customer by using thejob manager 50 to dispatch corresponding cloud engines 36 for preparing(e.g. advanced video coding) and publishing the digital media assets tospecific customers at 210.

Referring now to FIG. 14 there is shown a block diagram of an examplecertification system 160 in accordance with example embodiments. Thecertification system 160 may certify or sign a solution set, graph 28,component 24, blueprint 28 a, and so on (referred to generally herein ascomponents) with digital certificates 142, 132 to indicate acceptancethat the particular component will perform or carry out a function asexpected, by one or more user computing systems 140 associated with theparticular component and one or more component provider systems 130.FIG. 14 illustrates multiple user computing system 140, each associatedwith different media applications developed and deployed using system10. FIG. 14 further illustrates multiple component provider systems 130,each having provided resources such as one or more components for use bysystem 10 or one or more hardware resource used to execute computingapplications, for example.

A user computing system 140 may be any networked computing deviceoperated by a user of system 10 including a processor and memory, suchas an electronic tablet device, a personal computer, workstation,server, portable computer, mobile device, personal digital assistant,laptop, smart phone, WAP phone, an interactive television, video displayterminals, gaming consoles, and portable electronic devices or acombination of these. A networked device is a device capable ofcommunicating with other devices and components of system 10 andcertification system 160 through a communication network such as network152. A network device may couple to the communication network through awired or wireless connection. Similarly, component provider system 130maybe any networked computing device operated by a resource providerincluding a processor and memory

Network 152 may be any network(s) capable of carrying data including theInternet, Ethernet, plain old telephone service (POTS) line, publicswitch telephone network (PSTN), integrated services digital network(ISDN), digital subscriber line (DSL), coaxial cable, fiber optics,satellite, mobile, wireless (e.g. WMAX), SS7 signaling network, fixedline, local area network, wide area network, and others, including anycombination of these.

The user computing systems 140 and the component provider systems 130use digital certificates 132, 142 to indicate that they agree that aparticular component operates properly. That is, the digitalcertificates 132, 142 signify acceptance by both the user computingsystem 140 and the component provider 130 that a particular componentsatisfies a performance standard. For example, the digital certificate142 may sign a particular component when a user computing system 140activates a digital button “I agree” linked to a digital representationof a license agreement. It may be important to track and recordacceptance by users and providers to efficiently resolve disputes and toensure only accepted components are used in the computing applicationsso that the computing application functions as agreed. It may beimportant that the functionality performed by the application or thedeliverable (what the application creates or transforms) can be tracked,agreed upon, etc. A digital certificate may be an electronic file thatidentifies an individual or organization and electronically indicates tosystem 10 that the individual or organization accepts a component. Adigital certificate may be issued by system 10 or by a third partycertificate issuer. A digital certificate may have security measures toauthenticate the individual or organization associated therewith so thatthe digital certificate is not used fraudulently.

The certification system 160 may extend to multiple user computingsystems 140, each with a digital certificate 142 for signing components,blueprints, and so on. The certification system 160 uses digitalcertificates 132, 142 to create a ‘chain of trust’ between all aspectsof the system 10. Trusted components may create trusted graphs (whichmay include trusted or untrusted components). A graph may become atrusted graph when signed by multiple user computing systems 140 andprovider systems 130 to create trusted exchange of trusted graphs(blueprints) for use within a media application. That is, components maybe signed using digital certificates 132, 142, and the signed componentsmay be used to create graphs and blueprints. The graphs and blueprintsmay also be signed using digital certificates 132, 142. Those signedgraphs and blueprints may form part of a computing application, andsystem 160 may check to ensure all of the computing applicationcomponents are signed (i.e. accepted by a user and provider) prior toexecuting the computing application.

As an example, a user computing system 140 may use system 10 to developa media application involving a particular component and may test anddeploy the particular component to ensure it functions properly. Oncethe user computing system 140 agrees that the particular componentsatisfies a performance standard (i.e. functions properly) then the usercomputing system 140 can indicate acceptance using a digital certificate142 associated with the user computing system 140. The use of a digitalcertificate to indicate acceptance enables the system 10 toautomatically and efficiently track and check the acceptability of mediaapplications and components thereof. The user computing system 140signifies acceptance by signing the component with the digitalcertificate 142 associated with the user computing system 140.Similarly, the component provider 130 signifies acceptance by signingthe component with a digital certificate 132 associated with thecomponent provider 130.

The license server 42 includes a certificate component matrix 146 whichmanages records relating to digital certificates 132, 142. Inparticular, a record links the digital certificates 132, 142 and theaccepted resources, such as components, graphs, computing applications,hardware resources used to execute computing applications, and so on. Acomponent may be used in multiple computing applications associated withdifferent user computer systems 140, where each user computer system 140is associated with a different digital certificate 142. Accordingly, thecertificate component matrix 146 may include multiple records associatedwith the same component, where each record links the component to adifferent digital certificate 142 associated with a different usercomputer system 140.

In accordance with some embodiments, the computing application isdeployed internally by system 10 or externally by a remote computingsystem. For example, the remote computing system may be cloud basedinfrastructure. The remote computing system may be operated by system 10or may be operated by a third party, for example. A cloud engine 36 or acloud agent 34 may query the license server 42 to ensure that acomponent has been signed by both digital certificates 132, 142 beforeexecuting a component at runtime as part of a media applicationassociated with the user computing system 140. If the component has notbeen signed by both digital certificates 132, 142 then the cloud engine36 or the cloud agent 34 may not execute the component and may providean error message requesting that the component be signed by the digitalcertificates 132, 142. The cloud engine 36 or the cloud agent 34 maysubmit a query that includes both the user computer system 140associated with the media application and the component. The cloudengine 36 or the cloud agent 34 is operable to verify that the relevantuser computing system 140 has accepted the component, as a component maybe associated with a different user computing system 140. Further, thecloud engine 36 or the cloud agent 34 is operable to verify that thecomponent provider 130 has accepted the component before deployment.

Further, the license server 42 may include a record database 150 whichstores a record or report each time the resource operates successfully,such as when a component successfully executes within the computingapplication, when a signed hardware resource successfully executes thecomputing application, and so on. The record establishes that theresource operated successfully (e.g. a component or blueprint executedsuccessfully) in the event of a dispute between the user computingsystem 140 and the component provider 130. The license server 42 maygenerate summary of all records associated with a resource for provisionto the component provider 130, system 10 or user computing system 140.The summary provides a mechanism to notify the component provider 130 oruser computing system 140 that the resource is or is not successfullyoperating.

Using certification system 160, system 10 is operable to supply somesigned components (signed with digital certificates) to guarantee acertain level of functionality. A signed component may establish trustregarding performance and functionality.

For example, a gas company may operate user computing system 140 and maycreate a computing application (including a blueprint 28 a, graph 28,components 24, and so on) which contains some signed components requiredto perform a particular job, such as a component configured to performjob data mining on a large database of information about potential drillsites, geophysical information for each of these sites, potential risksassociated with each site, costs, government limitations, environmentalliabilities, and so on. The company may not have the processingresources to perform the computations of the computing application in arequired timeframe and may use a remote computing system, such as acloud based infrastructure for example, to execute the computingapplication in order to perform the computations. The company may want aguarantee of a performance level in relation to the execution of thecomputing application by the remote computing system.

The system 10 may engage third parties, such as a component providersystem 130 (e.g. vendor of components 24), to provide the remotecomputing system, such as the cloud based infrastructure. The system 10and component provider system 130 may be associated with a service levelagreement that guarantees performance of the remote computing systemprovided by the component provider system 130. In order for the gascompany operating the user computing system 140 to trust system 10 torun their computing applications there may be a chain of trustestablished between the component provider system 130, the system 10,and the user computing system 140. Accordingly, there may be more thantwo digital certificates signing the computing application (orcomponents 24, blueprints 28 a, graphs 28 thereof). System 10 may useits own digital certificate to sign to the computing application toguarantee that it functions exactly as the gas company associated withthe user computing system 140 requires. In addition, the user computingsystem 140 may use their digital certificate 142 to sign the computingapplication and the component provider 130 (which may also be referredto as a service provider as it may provide the cloud computing resourcesin this example) may use their digital certificate 132 to sign the mediaapplication. In effect, instead of offering remote computing systemresources (such as raw cloud resources for example), the system 10 maybe viewed as offering a “Workflow As A Service” in some examples. Thesystem 10 may not know exactly what job the media application isperforming only that it functions in the remote computing systeminfrastructure properly. The digital certificates 142, 132 provide achain of trust and acceptance between the parties, a security mechanismand a validating reference point that the parties can feel confidentabout. Accordingly, in this example, the gas company operating the usercomputing system 140 signs the application (or blueprint, graph,component thereof) with their digital certificate, the system 10countersigns the media application with their digital certificate, andthe component provider system 130 signs the media application with theirdigital certificate. The system 10 is operable to only execute theblueprint only if all digital signatures (made by the digitalcertificates) are valid. Security may be derived from the digitalcertificates and chain of trust established thereby, as only signedcomponents, blueprints and graphs may be executed by system 10 in someembodiments.

As system 10 generates reports each time the computing application (andblueprints, graphs, and components thereof) successfully execute andthose reports are stored by license server 42 in the record database150. A results summary may be generated and transmitted to the usercomputing system 140. The chain of trust is maintained so that the gascompany operating the user computing system 140 can trust the results ofthe media application, third party service, etc. and the fact that thedata and blueprints have been protected throughout the entire process.

Other potential types of computing applications and contexts includevoter registration, integrity management, and vote tally applications.

For example, a computing application may include a blueprint defining aworkflow at place at each voting station. The computing application maybe a trusted application using trusted (i.e. signed) components, runningon a trusted system 10. The user (voter) registers their data and signsthe data using their digital certificate and the system 10 adds a userspecific certificate to the data. At voting time, the trusted user datais interjected into the trusted workflow and the vote is recorded bysystem 10. All of the voter stations involved send trusted data into theremote computing system hardware infrastructure executing the computingapplication where trusted processes are running inside of a trustedenvironment. The remote computing system solution provider (e.g.component provider system 130) countersigns the computing applicationimplementing the workflow with their digital signature, the data comingback is secure (i.e. signed), the public and government can trust thesystem 10 because it is secure (i.e. signed), and the results aretrusted because of the nature of the system 10. The signing of acomponent may involve encrypting the component to enhance security.

The use of digital certificates in the described example embodimentsdiffers from a traditional https style of key exchange. For an httpssystem, when a user accesses a website, say a bank, the user can trustthe bank because the certificate at the bank website is certifiedagainst that particular system. In a remote environment, such as a cloudenvironment, where the system providing the remote hardwareinfrastructure may be unknown, and using the described embodiments, thedigital certificates may be used to secure the process implemented bythe computing applications. The described embodiments may use digitalcertificates to sign the entire media application, including theindividual components and entire workflows implemented by blueprints 28a, and even multiple stages of workflow.

As another example, a media company may sign a media applicationprovided by system 10 as a Workflow As A Service. The media company mayoperate the user computing system 140 to access system 10 and developblueprints (workflow) for the media application. System 10 or a thirdparty, such as a media company association, may certify that theworkflow (blueprint or media application) is secure when all involvedcompanies sign the blueprint using their digital certificates. Thesignatures makes all parties feel protected, and ensures with some levelof credibility that the media will be secure, the processes will besecure and operate as expected, and the chain of trust is secure.

A process flow associated with the certification system 160 may includethe following steps in accordance with an example embodiment.

A user computing system 140 may be provided, via network 140, withaccess to the system 10, which includes a development framework 12 witha software development kit 20, components 24, data containers 56, pins,graphs 28, blue prints 28 a and so on, in order to construct a computingapplication. The software development kit 20 may be used to define thecomponents 24, data containers 56, pins, graphs 28, blue prints 28 a,and solution sets (each generally referred to as components). Eachcomponent defines a computing processing mechanism for processing datacontainers 56 of data at application runtime. Each graph 28 or blueprint28 a comprises a reference to a solution set of components, where thesolution set of components is a set of particular versions ofcomponents.

The user computing system 140 may be provided, via network 140, withaccess to the visual design subsystem 30 configured to define and outputgraphs 28 and blueprints 28 a in order to develop computingapplications. The visual design subsystem 30 is operable to arrangecomponents into functional blocks and define specific orders ofoperation for the functional blocks. The user computing system 140 mayuse the visual design subsystem 30 in order to define solution sets forblueprints 28 a and graphs 28 using interface 10.

The user computing system 140 may be provided with a digital certificate142 associated therewith. The certificate 142 may be provided by system10, and in particular by license server 42. The certificate 142 may alsobe provided by a third party certificate provider.

The component provider 130 provides one or more resources, such ascomponents to system 10 for use by the user computing system 140 todevelop components and computing applications. Other example resourcesinclude signed computing applications and hardware resources to executecomputing applications. The component provider 130 is provided with acertificate 132 associated with the provided components. The certificate132 may be provided by system 10, and in particular license server 32.The certificate 132 may also be provided by a third party certificateprovider.

The user computing system 140 may use the component provided by thecomponent provider 130 for one or more of their media applications. Asdescribed herein, the user computing system 140 may require a license touse the component, as managed by the license server 42 and the licensepool 44.

The user computing system 140 may be provided, via network 140, withaccess to the deployment subsystem 14 for deploying the computingapplications including the component. As described herein, thedeployment subsystem 14 includes a repository 32, cloud agent 36, cloudengine 34, and other modules. The computing application may identifygraphs 28, blue prints 28 a, compound components 26, and components 24,including the component provided by the component provider 130. Therepository is configured to store graphs 28, blueprints 28 a, andcomponents 24 for loading at application runtime.

The user computing system 140 may use the deployment subsystem 14 totest and deploy the component provided by the component provider 130. Ifthe user computing system 140 is satisfied that the component providedby the component provider 130 functions as expected then the usercomputing system 140 may accept the performance by signing the componentusing the digital certificate 142 associated therewith. The licenseserver 42 receives the digital certificate 142 from the user computersystem 140 via network 152 and creates a record in the certificatecomponent matrix that links the signed component with the digitalcertificate 142. Similarly, the component provider 130 may accept theperformance by signing the component using the digital certificate 132associated therewith. The license server 42 receives the digitalcertificate 132 from the component provider 130 via network 152 andupdates the record in the certificate component matrix to also link thesigned component with the digital certificate 132.

A cloud engine 36 provides a running environment for the mediaapplications (and the graphs 28 and blueprints 28 a thereof) andexecutes the graphs 28 and blueprints 28 a at runtime to instantiate themedia application. The cloud agent 34 controls the cloud engine(s) 36.At runtime, the deployment subsystem 14 dynamically constructs anddeploys a computing application by sending a request at runtime to therepository 32 for the graphs 28, blueprints 28 a, compound components26, and components 24 identified in the media applications. Thedeployment subsystem 14, and in particular the cloud engine 36 or thecloud agent 34, is operable to query the license server 42 at runtime toensure that the component of the computing application has been acceptedby both the user computing system 140 and the component provider 130prior to executing the component and running the computing application.The license server 42 is operable to respond to the query by checkingthe certificate component matrix 146 for a record that links thecomponent to the user computing system 140 and the component provider130. If the component has not been accepted an error message may beprovided requesting acceptance.

Each time the component of the computing application successfullyexecutes the cloud agent 36 or the cloud engine 36 may provide a recordor report of the successful execution to the license server 42.

The job manager 50 is operable to store the record of successfulexecution in the record database 150 in association with the component,the user computing system 140 or the component provider 130. That way,if a dispute arises in relation to the operation of the component thejob manager 50 can provide a copy of the record to establish that thecomponent did or did not execute successfully to resolve the dispute.The license server 42 may know whether or not specific technologyresources are in use, may not know whether or how the technologyresources used was actually successful.

In accordance with some embodiments, system 10 is operable to providesupport for mixed architectures. This may provide increased flexibilityas typically a process needs to be compiled for the same architecturebinary. For example, a 32 bit CODEC library would typically have to runon a 32 bit context, and typically could not run in a 64 bit context orwith a 64 bit library. In accordance with some embodiments, system 10 isoperable to develop and deploy an application instance which combinescomponents 24 written for both 32 bit and 64 bit architectures. System10, and in particular cloud engine 36 a, is operable to detect whether aparticular media application has been developed using both components 24for different architectures, such as components 24 for 32 bitarchitectures and components 24 for 64 bit architectures, for example.If so, system 10 is operable to create a separate process space orinstance for each context and handle inter process communications usingmapping and a shared memory. For example, the system 10 is operable tocreate a 32 bit architecture process instance and a 64 bit architectureprocess instance and manage communications between the processinstances.

Referring now to FIG. 16 there is shown a block diagram of a mixedarchitecture in accordance with example embodiments. A graph 28 maycontain components 24 developed for different architectures, such ascomponents 24 for 32 bit architectures and components 24 for 64 bitarchitectures, for example. System 10, and in particular cloud engine 36a, is operable to detect whether a particular graph 28 includescomponents 24 for different architectures. In this example, graph 28contains components 24 a, 24 b, 24 c, 24 d developed for 32 bitarchitectures and further contains a subgraph 28′ with componentsdeveloped for 64 bit architectures. System 10 is operable to create aseparate process space or instance of graph 28′ for the 64 bit contextand handle inter process communications using mapping and a sharedmemory to receive and provide input and output. The subgraph 28′ may runout of process for example.

In accordance with some embodiments, system 10 is operable to provideselective scalability, through dynamic provisioning, and deploymentbased on the individual workload of a component 24, group of components24, or entire graph 28/blueprint 28 a. System 10 is operable to analyzethe runtime processing of a blueprint 28 a and break down the blueprint28 a into separate running graphs 28 based on derived knowledge of theprocessing overhead (e.g. bottlenecks) of particular components whichexist in a particular workflow. System 10 is further operable to isolatethose component(s) 24 and partition the blueprint 28 a into multiplegraphs (e.g. create new graphs 28) on the fly which can exist on aseparate host system (or the same host system) while maintaining thecommunication and integrity of the original workflow/blueprint 28 a. Forexample, system 10 is operable to use a separate host system with moreresources to process to overhead/bottlenecks and manage data flowsbetween. The modular nature of components 24 and graphs 28 enable system10 to partition a graph 28 into multiple graphs and run them separatelyon different computing resources.

Referring now to FIG. 15 there is shown a block diagram of a graph 28partitioned into two graphs 28, where each is run on a separate hostsystem 300, 302. System 10 is operable to identify a processingbottleneck (i.e. graph 28′) in a graph 28 running on a host system 300,isolate the components 24 associated with the processing bottleneck, andcreate a new graph 28′ on the fly which can exist on separate hostsystem 302. The separate host system 302 provides additional resourcesto process graph 28. System 10 is operable to manage communications anddata flow (e.g. via a shared memory) between the host systems 300, 302and graphs 28, 28′.

In accordance with some embodiments, system 10 is operable to providesecurity through dynamic re-location of graphs 28 and components 24 thatmake up a particular computing application. The process is similar to asdescribed above in relation to selective scalability except that a graphis partitioned for the purpose of increased security, as opposed to thepurpose of isolating a processing bottleneck. For example, securitymodule 46 may interact with license server 42 to determine whether aparticular graph 28 for a media application refers to components 24 (orembedded graphs 28) that are signed. If so, system 10 may partition thegraph 28 for the media application by isolating the signed components 24for increased security by obfuscating the original footprint of themedia application. The partitioned components may then run on a separatehost system (similar to that shown and described in FIG. 15). System 10is operable to further limit access to the running footprint of aparticular blueprint 28 a and relocate sections of the blueprint 28 aonto different hosts in a dynamic fashion. This may be viewed as‘scrambling’ the application footprint at the functional level making itharder to find and compromise. For example, an attacker may focus on onehost to monitor and listen, so splitting a process onto multiple hostsmay make it more difficult for an attacker to monitor and tamper withthe program. The original functionality is maintained as system 10manages communications and data between the multiple host systems.Security is maintained as the process of creating the new sub-graphs 28may involve automatic injection of secure messaging components (e.g.encryption). As an extension of this, system 10 is further operable tocreate multiple instances of each new sub-graph 28 and to randomize thedata paths through the running components 24 of the sub-graphs 28.Further, system 10 may be operable to maintain a set of potential hostsystems and randomize selection of a subset of those host systems onwhich to run the program, so that the host systems are also randomlyselected.

In accordance with some embodiments, system 10 is operable to controlaccess to resources using virtual priority management. That is, system10 is operable to tune a media application to control priority ofprocessing, parallelization and threading. At runtime, system 10 isoperable to manage the execution of a component 24 in such a way as tomake it process faster or slower than normal. There may be an imbalanceof resource utilization between components 24 in a media application,and system 10 is operable to manage the processing prioritization of aparticular component while other components 24 are prioritizedindependently. For example, if a more important component 24 is runningthen system 10 is operable to manage, control, manipulate or throttle(i.e. slow down) another component that may be consuming a larger amountof resource until the more important component 24 has completed itsprocessing.

As an illustrative example, the virtual priority management may beimplemented using a virtual clock as a mechanism to control priority. Avirtual clock is one example and the implementation could be done anumber of different ways.

As noted above system 10 is operable to limit resources allocated to aparticular component 24. For example this may be limiting componentaccess to a thread pool, memory, or other mechanism. System 10 maythrottle the data while not touching anything. An example may be asource component such as a complex signal generator. The signalgenerator may be the first component in the pipeline and may generateframes faster than they can be consumed but while doing so can also usesome amount of CPU. If system 10 decides at runtime to limit the CPUactivity the system 10 may simply trigger the signal generator lessoften. This may not require any manipulation of threads, memory, orsource data packets. Another example may be something at the end of thepipeline that is designed to send out notifications or updates but isalso blocking while doing so. The component may send email notificationsand the act of sending those notifications takes longer than the rest ofthe pipeline does in actual processing. System 10 may limit the numberof notifications by throttling the packets that normally trigger thecomponent to start execution.

In accordance with some embodiment, components 24 may be self-containedand isolated from a dependency point of view. The entire dependency setof a component may be self-contained, being specified and packaged inthe component 24 distribution unit (plugin). The component 24dependencies may also be isolated, referring exclusively to the specificcomponent 24 and version(s) thereof they depend on. Thus the system 10may be able to realize complex workflows while resolving componentsdependencies without user intervention. Further the dependency isolationmay allow the system 10 to provide distinct behavior while executingdifferent solution sets (blueprints 28 a) built with the same components24, by isolating the different versions of these components 24 and theirdependencies.

As described herein, graphs 28 and blueprints 28 a are portable and maybe packaged to run anywhere there is a cloud agent 34.

In accordance with some embodiments, components 24 and graphs 28 maysupport promotion of properties and values. For example, if onecomponent 24 is embedded within another component 24, the innercomponent 24 may promote one or more of its properties to theouter-component 24. A user may pass expressions to components 24 tochange/promote properties. That is, properties may be reauthored as theyare promoted. Properties may be selectively promoted in that not allproperties need to be promoted together. Properties may be promotedwithout exposing the values, and without exposing the values of theproperties that are not promoted.

Embodiments have been described herein in relation to media applicationsas an illustrative example. The system 10 and methods described hereinmay be used to develop and deploy other type of software applicationsand are not limited to media applications, such as natural resourceapplications, voting applications, and so on.

Embodiments have been described here by way of example only. Variousmodification and variations may be made to these exemplary embodiments.

We claim:
 1. A system for dynamic development of computing applicationscomprising: one or more linked repositories storing blueprints, graphs,and components; development processor configured with a softwaredevelopment kit having a framework application programming interface anda component application programming interface; visual designerconfigured to provide a graphical interface to access the softwaredevelopment kit; the framework application programming interface todevelop, test and output at least one computing application realized bya blueprint of the blueprints in the one or more linked repositories,the blueprint used to instantiate at least one graph of the graphs inthe one or more linked repositories at application runtime, the at leastone graph representing a workflow of a set of components from thecomponents stored in the one or more linked repositories, the workflowdefining an arrangement of the set of components and connections therebetween using pins, the set of components for the plurality ofarchitectures, the framework application programming interface tocontrol loading and executing the at least one graph, and to providestatus to the at least one computing application, the at least onecomputing application to process at least one input data stream togenerate at least one output data stream; the component applicationprogramming interface to develop the components in the one or morelinked repositories, each component defining a computing processingmechanism for processing data containers of computing data, thecomponent application programming interface configured to write the setof the components for the plurality of architectures; and thedevelopment processor configured to detect that the at least onecomputing application includes the set of the components for theplurality of architectures, create a separate process space for eacharchitecture and handle inter process communications between the processspaces.
 2. The system of claim 1, wherein a graph delivers functionalitydefined by the components identified by the graph, and wherein ablueprint connects the functionality to a running environment.
 3. Thesystem of claim 1, further comprising a certification system to issue adigital certificate to sign the solution set of components to indicateagreement that the set of components of the solution set satisfy aperformance standard.
 4. The system of claim 1, wherein the visualdesign subsystem is configured to generate the graphical interface toarrange a set of the components into functional blocks, define specificorders of operation for the functional blocks, and define theconnections between the functional blocks using the pins to define theat least one computing application.
 5. The system of claim 1, wherein afirst component is in a first language and a second component is in asecond different language, wherein the first and second componentscomprise data and are operable to access the memory and data structures,and wherein the system further comprises a translation module operableto translate multiple languages into a common language by translatingthe first and second component data and how the first and secondcomponent are operable to access the memory and the data structures. 6.A computer-implemented method for dynamic development of computingapplications, the method implemented with at least one computer coupledto memory-stored executable instructions which, when executed by the atlast one com Inter perform the method, comprising: storing blueprints,graphs, and components at one or more linked repositories; configuringone or more processors to execute a command to provide a softwaredevelopment kit having a framework application programming interface anda component application programming interface; configuring the one ormore processors to provide a graphical interface to access the softwaredevelopment kit; developing, using the component application programminginterface, the components in the one or more linked repositories, eachcomponent defining a computing processing mechanism for processing datacontainers of computing data; configuring a set of the components for aplurality of architectures using the component application programminginterface; developing and outputting, using the framework applicationprogramming interface, at least one computing application realized by ablueprint of the blueprints in the one or more linked repositories, theblueprint used to instantiate at least one graph of the graphs in theone or more linked repositories at application runtime; instantiatingthe at least one graph of the graphs in the one or more linkedrepositories using the blueprint at application runtime, the at leastone graph representing a workflow of components from the componentsstored in the one or more linked repositories, the workflow defining anarrangement of the plurality of components and connections between thecomponents using pins; loading the set of components from the one ormore linked repositories using the at least one graph; and processing,using the at least one computing application, at least one input datastream to generate at least one output data stream.
 7. The method ofclaim 6, further comprising generating a digital certificate to sign theset of components to indicate agreement that the set of componentssatisfy a performance standard.
 8. A system for dynamic deployment ofcomputing applications comprising: one or more linked repositoriesstoring blueprints, graphs, and components, each component defining acomputing processing mechanism for processing data containers ofcomputing data; a software development kit having a frameworkapplication programming interface and a component applicationprogramming interface, the framework application programming interfaceto develop, test and output at least one computing application realizedby a blueprint of the blueprints in the one or more linked repositories,the component application programming interface to develop thecomponents in the one or more linked repositories, each componentdefining a computing processing mechanism for processing data containersof computing data, the component application programming interfaceconfigured to write a set of the components for a plurality ofarchitectures; at least one deployment processor for receiving a commandto deploy the at least one computing application to process at least oneinput data stream to generate at least one output data stream, thedeployment processor using the blueprint to instantiate at least onegraph of the graphs in the one or more linked repositories atapplication runtime, the at least one graph representing a workflow ofcomponents from the components stored in the one or more linkedrepositories, the workflow defining an arrangement of the set ofcomponents and connections between the components using pins; thedeployment processor configuring one or more cloud agents, and one ormore cloud engines, the cloud agent receiving the command to deploy theat least one computing application to process the at least one inputdata stream and, in response, instantiate the one or more cloud engineson a host system and provide a running environment for the one or morecloud engines; and the one or more cloud engines dynamically constructthe at least one computing application on the respective host system byrealizing requirements of the blueprint of the at least one computingapplication, the requirements identifying the at least one graph and thecomponents of the workflow and by sending a request to the one or morelinked repositories to load the blueprint, the at least one graph, andthe set of components on the respective host system; the deploymentprocessor configured to detect that the at least one computingapplication includes the set of the components for the plurality ofarchitectures, create a separate process space for each architecture andhandle inter process communications between the process spaces.
 9. Thesystem of claim 8, further comprising a license server, wherein thelicense server dynamically manages licenses and associates licenses withcomponents and graphs, wherein use of components and graphs atapplication runtime requires the associated license.
 10. The system ofclaim 8, further comprising a job manager, wherein the job managerdispatches blueprints and graphs to cloud agents based on availablelicenses managed by the license server.
 11. The system of claim 8,further comprising a security manager, wherein the security managerprovides for secure connections and communications between systemcomponents.
 12. The system of claim 8, further comprising a job managerconfigured to provide job and cloud engine dispatch, failover, trackingand reporting.
 13. The system of claim 8, further comprising acertification system to issue a digital certificate to sign the set ofcomponents to indicate agreement that the set of components satisfy aperformance standard.
 14. The system of claim 8, wherein a datacontainer defines a data type and a data object, wherein the data typeis metadata describing the data container and the data object maintainsraw data.
 15. The system of claim 8, wherein the cloud agent is providedto each user system to manage the local resources of the user system,wherein the cloud agents interact with cloud engines to instantiategraphs using blueprints.
 16. The system of claim 8, further comprising anormalization module operable to receive input data files and convertand parse the input data files into data containers for processing by agraph.
 17. The system of claim 8, further comprising a code signingmodule operable to digitally sign each component to associate at leastone of a developer and license with at least one component.
 18. Thesystem of claim 8, further comprising: a digital certificate associatedwith a component provider subsystem, wherein the component providersubsystem provides one or more components of the set of componentsidentified by the solution set; a digital certificate associated with auser computing subsystem, wherein the user computing subsystem isassociated with the at least one computing application, wherein thecomputing application involves a component provided by the componentprovider computing system; a license server configured to digitally signa component of the set of components by linking the component to thedigital certificate associated with the user computing subsystem and thedigital certificate associated with the component provider subsystem toindicate that the user computing system and the component providersubsystem accept performance of the digitally signed component; whereinat runtime prior to deploying each component the deployment subsystemqueries the license server to determine whether the component is linkedto the digital certificate associated with the user computing subsystemand the digital certificate associated with the component providersubsystem.
 19. The system of claim 8, wherein the deployment subsystemis further configured to partition a graph into two or more subgraphsand handle interprocess communications between the two or moresubgraphs.
 20. A computer-implemented method for dynamic development ofcomputing applications, the method implemented with at least onecomputer coupled memory-stored executable instructions which, whenexecuted by the at last one computer perform the method, comprising:storing blueprints, graphs, and components, each component defining acomputing processing mechanism for processing data containers ofcomputing data; configuring one or more processors to execute a commandto provide a software development kit having a framework applicationprogramming interface and a component application programming interfacedeveloping and outputting, using the framework application programminginterface, at least one computing application realized by a blueprint ofthe blueprints in the one or more linked repositories; developing, usingthe component application programming interface, the components in theone or more linked repositories, each component defining a computingprocessing mechanism for processing data containers of computing data,the components including a set of the components fora plurality ofarchitectures; providing at least one deployment processor for receivinga command to deploy at least one computing application to process atleast one input data stream to generate at least one output data stream;realizing the at least one computing application by a blueprint of theblueprints in the one or more linked repositories; instantiating, usingthe blueprint, at least one graph of the graphs in the one or morelinked repositories at application runtime; using the at least one graphto represent a workflow of components from the components stored in theone or more linked repositories, the workflow defining an arrangement ofthe plurality of components and connections between the components usingpins; configuring, using the deployment processor, one or more cloudagents, and one or more cloud engines; receiving, at the cloud agent,the command to deploy the at least one computing application to processthe at least one input data stream and, in response, instantiating theone or more cloud engines on a host system and provide a runningenvironment for the one or more cloud engines; dynamically constructing,using the one or more cloud engines, the at least one computingapplication on the respective host system by realizing requirements ofthe blueprint of the at least one computing application, therequirements identifying the at least one graph and the components ofthe workflow and by sending a request to the one or more linkedrepositories to load the blueprint, the at least one graph, and the setof components on the respective host system; and detecting that the atleast one computing application includes the set of the components forthe plurality of architectures and creating a separate process space foreach architecture and handle inter process communications between theprocess spaces; processing the input data stream as a plurality of datacontainers using the plurality of components to generate the output datastream, the plurality of data containers flowing between the componentsof the workflow using the pins and the inter process communicationsbetween the process spaces.
 21. The method of claim 20, furthercomprising: generating a digital certificate associated with a componentprovider subsystem, wherein the component provider subsystem providesone or more components of the set of components; generating a digitalcertificate associated with a user computing subsystem, wherein the usercomputing subsystem is associated with the at least one computingapplication, wherein the computing application involves a componentprovided by the component provider computing system; providing a licenseserver configured to digitally sign a component of the set of componentsby linking the component to the digital certificate associated with theuser computing subsystem and the digital certificate associated with thecomponent provider subsystem to indicate that the user computing systemand the component provider subsystem accept performance of the digitallysigned component; receiving, at a license server, acceptance of thecomponent provided by the component provider subsystem in the computingapplication associated with user computing system by receiving thedigital certificate from the user computing subsystem and the digitalcertificate from the component provider computing system; linking, atthe license server, the component provided by the component providersubsystem in the computing application associated with user computingsystem to the digital certificate from the user computing subsystem andthe digital certificate from the component provider computing system;and at application runtime prior to deploying each component, queryingthe license server to determine whether the component is linked to thedigital certificate associated with the user computing subsystem and thedigital certificate associated with the component provider subsystem.